The hunt for a new drug is a search through billions of molecules

Biotech & Longevity 4 min 8 sources

Pharma is pouring money into AI and supercomputers to sift a chemical space too vast to check by hand. New tools this week narrow the haystack rather than search it faster - while the drugs that survive that hunt still mostly fail downstream.

Key takeaways

Finding a drug means searching a chemical space of tens of billions of molecules, so pharma is spending heavily on AI and supercomputers to sift it.
The winning tools narrow the haystack cleverly - ranking billions so a lab only tests the top few - rather than just checking faster.
A smart search only fixes the front of the pipeline; the drugs that survive it still mostly fail in human trials, as this week's mix of approvals and flops shows.

The search is the hard part

Finding a new medicine is, at bottom, a search problem. Somewhere in the set of possible molecules is one that binds the right target, stays stable in the body, and can be made at scale. The trouble is how many molecules there are to check.

This week two of the industry’s biggest bets on that search went public. Bristol Myers Squibb said it will build what it called the “most powerful AI factory in life sciences” with Nvidia, deploying a supercomputer to speed its drug research [1]. It extends a three-year collaboration and sits alongside BMS deals with Anthropic’s Claude and Microsoft [1]. “We are beginning to see it pay off in our pipeline,” said BMS technology chief Greg Meyers [1].

The reason for the spending is scale. A single make-on-demand library from the supplier Enamine now lists tens of billions of molecules a chemist could order [2][3]. Physically testing them in a lab reaches maybe a million - the tiny slice a screening centre can actually hold [3]. “Scientists explore vast quantities of possible molecules, looking for the rare few,” an AstraZeneca R&D account explained this week [4]. The number of combinations “far exceeds what any human team can systematically explore” [4].

Narrowing beats brute force

The interesting move is not searching faster - it is searching less. Almost every new tool this week is a way to throw most of the haystack out before testing anything.

A German group screened 19 billion molecules from Enamine’s virtual library on a “neuromorphic” chip called SpiNNaker2 - hardware wired to imitate a brain [3]. It ran the sift about four times faster than a comparable Nvidia system and used roughly 86% less energy [3]. But the point is the sift itself: a computer ranks billions so a lab only ever touches the top few.

A US team went further and asked whether the shortlist is any good. Their pipeline, called REAL-M, uses the shapes of known protein-molecule contacts to guide which candidates to pick [2]. Tested on one brain receptor, 28 of 30 predicted blockers actually worked in a cell assay, and three held up in live zebrafish [2]. That hit rate is the prize. Most virtual screens spit out thousands of computer-approved molecules that then fail at the bench [2]. A good score on a screen does not mean the molecule works in a living thing [2]. AstraZeneca frames it as a “build-measure-learn loop”: the AI ranks, the lab tests only the top, the results retrain the ranker [4].

The survivors still mostly fail

A clever search narrows the front of the pipeline. It does nothing for the long, failure-prone middle - the trials where most of the survivors still die.

The week’s approvals and misses show it. GSK won an early US approval for a lung-cancer drug, Jideytro, two months ahead of the regulator’s own deadline [5]. It came just weeks after GSK bought the treatment in a $10.6 billion deal [5]. Eli Lilly said new data on its “triple-G” obesity drug retatrutide clear it to file for approval [6]. The drug drove 22.6% average weight loss in one trial, though the heart-health benefit is still unclear [6]. And Kolon’s cell-based gene therapy for arthritic knees failed its phase 3 trial outright [7]. A phase 3 trial is the large, final human test before approval; failing it usually ends the program.

None of those outcomes was set by how the molecule was found. The AI-narrowed search only decides which candidates enter the funnel - biology decides which come out.

A vaccine built in eight weeks

The week’s quieter story shows the same search logic running in reverse - narrow first, then move fast. In Oxford, the first volunteer received an experimental Ebola vaccine built in just eight weeks, using the same platform as the Oxford-AstraZeneca Covid shot [8]. Ed Hunt, a 37-year-old, is the first of 50 healthy volunteers in the trial [8]. He signed up after seeing reports of the Ebola outbreak in Democratic Republic of Congo [8].

An eight-week build is possible because the hard search was done years ago: the platform was already chosen, so only the target had to change. It is an early safety trial in healthy people, not proof the vaccine protects anyone - that comes later, if at all. But it is a reminder of what the whole AI-search push is chasing: the day the slowest step in medicine is no longer the looking.

02 · Lesson · why it matters

Why some problems are hard just because they are huge

A search can be hopeless not because any single step is hard, but because there are astronomically many things to check - and the only escape is a clever way to look at fewer.

Two kinds of hard

We usually picture a hard problem as one where each step is difficult. Lifting something heavy. Solving a knotty equation. Doing a tricky surgery.

But there is a second kind of hard, and drug discovery is the clearest example. Each single step is easy. Take one molecule, test whether it sticks to the target, see if it works. A lab does that thousands of times a day. The problem is not the step. The problem is how many steps there are.

A make-on-demand chemical library now lists tens of billions of molecules a chemist could order. The number a lab can physically test is around a million. So the trouble is not that any one test is hard. It is that the thing you want is one grain hidden in a beach.

Faster is not the fix

The instinct, when a search is too big, is to search faster. Buy a bigger computer. Run more assays. This week a supercomputer and a brain-imitating chip both went to work sifting billions of molecules.

But speed alone loses this race. If the haystack is ten thousand times bigger than what you can check, doubling your speed still leaves you checking a rounding error of it. You feel busy and you find nothing.

The molecules that matter are rare, and they do not announce themselves. Testing at random, you would spend nearly all your effort on dead ends - molecules that were never going to work - just to stumble onto the few that do. The size of the space, not the difficulty of the test, is what defeats you.

The real move is to look at less

So the actual work is not searching harder. It is throwing most of the haystack away before you look.

That is what every clever tool this week really does. A ranking model reads billions of molecules and hands the lab only the top few. A method called REAL-M uses the shapes of proteins already mapped to guess which candidates are worth trying - and when researchers tested its guesses, 28 of 30 actually worked. That hit rate is the whole game. A good filter turns an impossible search into a cheap one.

This is the pattern, and it runs far past medicine. Finding a chess move, a suspect, a gene behind a disease, a password - all are searches through a space too big to check. In every one, the winners are not the ones who check fastest. They are the ones who find a reason to ignore almost everything.

The danger in the filter

There is a catch, and it is the honest part. A filter that saves you from the vast search can also throw away the very thing you were looking for.

The same tool this week that caught 28 of 30 winners also runs the risk that some real winner carries a feature the filter was told to skip. Tighten the filter to save more effort and you find fewer true hits. Loosen it and you are back in the whole ocean. There is no filter that is both cheap and perfect. Every narrowing is a bet that the thing you want looks the way you expect it to.

That bet is why so much of medicine’s map has blank regions. The corners of the molecular space that get searched are the ones someone had a reason - and a budget - to search. A rare disease with few patients is not a harder chemistry problem than a common one. It is a corner of the same vast space that no filter was ever pointed at, because no one could pay to point it there. The space is neutral. The choice of where to look is not.

What the size of the space asks of us

It is worth sitting with how strange this kind of difficulty is. Nothing about any single molecule is beyond us. We can make it, test it, understand exactly why it did or did not work. The wall is only that there are too many, and we are small in front of the number.

So the honest posture is not confidence that we will find the cure, but respect for how much of the space no one has looked at. The drug that would have worked may already be orderable, sitting unexamined among ten billion others, filtered out years ago by a guess that seemed reasonable. We are not above the haystack, sorting it from a height. We are inside it, holding a small light, choosing where to point it - and mostly choosing not to look.

03 · Lab · your turn

Search the Haystack

Hunt four working molecules among 400, and feel how narrowing the search cleverly beats testing everything - and what a filter can cost you.

04 · Hope · carry this

The haystack is vast, but every clever filter we build means one more corner of it finally gets a look - and somewhere in the unsearched billions, the next cure is already waiting to be found.