There are a number of recent comments (here, here, and here) about the failures of AI in drug discovery. While the underlying facts are true, and ‘AI-generated’ drugs have so far not lived up to their hype and expectations in the clinic, the conclusions are not quite right. The articles show that there is a fundamental misunderstanding of how AI (I will call it ‘machine learning’ going forward) is used in the pipeline today and why certain claims on both sides (AI drug discovery companies as well as critics) won’t ever be testable or realised — and hence can’t stand closer scrutiny.
Machine learning is not a panacea for drug discovery. And will likely never be (at least not in the next 50–100 years). The reason is that the drug discovery process is way too complicated and human biology is way too complex and diverse. This means that we will unlikely be able to collect sufficient amounts of data to computationally design and predict the outcomes of clinical trials, ignoring the fact that the technology of today might not even be able to capture all relevant biological parameters. This is in stark contrast to some claims that have been made over the past few years. It is fair to say that a number of these claims, albeit unfounded, have led to unrealistic expectations. But does that mean that machine learning does not have any impact?
Currently machine learning is mainly used to bring new drug candidates to the preclinical stage. This means that machine learning helps either with the identification of novel targets, finds better hits, or optimises the hits to turn them into early leads, then late leads and possibly even into a development candidate. This means that it helps us find molecules that satisfy a molecular target profile or perhaps a complex target product profile (if we get lucky and the preclinical tests are all positive, for example through PK/PD or early toxicity predictions). If we optimise for molecular product profiles or target product profiles not fundamentally different to historic ones, then the resulting molecules will unlikely be fundamentally different to historic ones. The question then is why would compounds be more successful?
Knowing what to optimise for is most often the biggest question. Sometimes we know what we’d like but don’t know how to optimise for it, or (more often) we simply do not yet know what to optimise for since we don’t have a good understanding of disease biology!
Some AI-enabled drug discovery companies go after new targets — possibly without the proper (clinical) validation. If targets are not new and have enough evidence to be clinically relevant, then AI can’t help us with that. Even the best compound might not have the desired effect in the clinic.
In most instances, however, the targets are not new. Looking at Exscientia’s pipeline, specifically their 5-HT1a agonist, it’s nothing new at all as already commented on by Derek Lowe. In fact, a quick search on GlobalData reveals 516 molecules at various stages and a structural analysis also revealed that the compound shares its shape with haloperidol, a frequently used first-generation antipsychotic agent, which the FDA approved in 1967. This makes sense from a machine learning perspective, since there is a lot of training data available (see the number of ligands here) and machine learning can hence more likely be used.
However, while this data can help training machine learning models to quickly find hits (molecules that stick to the target) and potentially some optimised leads, it won’t necessarily increase the overall probability of success in clinical trials — which means there was still a 90–95% chance that the drug would fail (and it did).
In order to increase the chances of success, we need to optimise the compounds to satisfy requirements that are driven by a better understanding of the disease or due to novel insights that emerged from previous failures. Without that, we cannot expect any new drug to be superior to previous ones other than by chance. To repeat Derek’s opinion, there is simply not enough reliable information to feed into machine learning models to allow us to predict what will happen against conditions like OCD, depression, anxiety, and many others. And that is the problem. Drugs fail at every stage for some expected or unexpected reasons. Some are due to biology, some due to technical issues, some are due to regulatory changes, and some might be epidemiological. Sometimes we are not picking the right target, because our biochemical understanding of the disease state is wrong and/or incomplete. But they can equally fail due to unexpected toxicity, and the situation with tox is the same as for CNS efficacy: no matter how good the machine learning models we have — we just don’t know enough to use it to tell us whether we are going to run into such problems.
But does this mean that machine learning is useless?
Of course not. While choosing the wrong target is often the problem, resulting in a lack of efficacy in clinical trials, there are many reasons for failure. Toxicities, for example, are one of the other major reasons — as was the case in many T-cell engagers. Here on-target off-tumour effects have commonly led to dose-limiting toxicities.
By designing molecules that take these learnings into account we can overcome past challenges and design novel, more efficacious, and safer therapeutics. Once we have such novel insights (e.g. a better understanding of the disease biology or nature of toxicity), and we have good (translational) models to early test for this, then we can use machine learning to help us with the generation and optimisation of these new therapeutics. In fact, we can screen much larger (chemical/biological) spaces using machine learning and this might help us find therapeutics that satisfy the more complex requirements of today’s therapeutics. And perhaps we wouldn’t have been able to identify these candidates only 10, 20, or 30 years ago. It should also be noted that a good fraction of best-in-class drugs are not first-to-launch but take into account learnings from previous generations. The flip side here is, of course, that new modalities, formats, etc., or more broadly speaking any new biology or chemistry might come with new risks — the unknown unknowns. Here, again, machine learning won’t help.
Now, turning to the question of ‘how can we prove that AI works?’.
As Patrick Schwab already said before — we can only assess the benefits of using machine learning at each stage. The reason for this is that we can establish clear benchmarks that allow for comparison. Requiring a full head-to-head comparison is an impossible task. For example, if we would want to compare clinical trial success rates of AI-enabled programs versus non-AI-enabled ones, it would require at least 150 clinical trials to observe a statistically significant difference, assuming that we have a 5% chance of success for the conventional ones and a 10% chance for the AI-enabled ones. This is not a fundamentally new insight, but it just reinforces that this will take a lot more trial outcomes than we have seen so far. Further to this, things do not become easier. The low hanging fruits have been plugged, so comparing new trials against historical ones is not necessarily a good metric as well.
Beyond this, it is even hard to formulate what is AI-enabled and what is not. I would argue that pretty much every drug discovery company today is using some form of machine learning. And the reason for this is that it works for many areas well. Whether we predict binding, ADME properties, T-cell activation, or developability characteristics — both in biologics and small molecules, machine learning (or more broadly speaking statistical modelling) is now commonly used (see Figure 1). And companies who don’t use any form of it are certainly falling behind.
Now, I agree that finding better candidates is key. However, saving a few months off the process across a large portfolio of say 20–30 programs can still result in massive cumulative savings and benefits. Assuming, for example, that we would be able to reduce the time of going from target to an optimised lead by 50%. From a cost and time perspective, this might enable us to run twice as many programs and then ultimately select the best performing ones to progress into the clinic.
So where does this leave us with AI in drug discovery today?
Saying a drug needs to be fully AI designed, i.e. all steps need to be done fully automated and autonomously is the same as requiring driving to be fully autonomously. While we still don’t have fully autonomous driving cars today, I would strongly argue that the automation of driving has not been a failure. Most people nowadays use lane assist, cruise control, or parking sensors very happily, and the overall driving experience has become a lot better and safer. The same could be said about machine learning and the different steps in drug discovery. Most companies are using machine learning very productively across all stages of the drug discovery process, and those not using AI (machine learning) are falling behind.
The vision is that machine learning will enable individual modules within the drug development process to be better or more efficient. These machine learning modules will need to work in unison with traditional approaches — both experimental and computational — in order to deliver programs that are most likely to succeed. And combining this with brilliant scientists and novel insights, will then lead to higher overall probabilities of success in clinical trials.
Integrating machine learning within the pipeline and existing workflows is key to reducing costs and timelines and being able to find better therapeutics in the future.
There is no simple panacea like a generative model that spits out a guaranteed drug. But perhaps it’s our expectations that need to be reshaped, because although machine learning cannot guarantee clinical success, it can certainly reduce some of the risks associated with drug failure.
In the end this shows one thing — which doesn’t come as a surprise: Drug discovery is hard. And it requires strong teams working closely together to enable high quality science. Equipping the best scientists with the best tools is the only way to deliver more and better therapeutics to patients who are waiting for these.