Your Models Are Learning Chemist Sociology, Not Chemistry

The claim. Leash Bio's CEO Ian Quigley argues that small-molecule drug discovery is being held back by weak data, not weak models. The company built DNA-encoded libraries (DELs) that screen millions of molecules against thousands of protein variants at basement-lab cost. He claims 2,000 protein constructs, roughly 5,000 screens, 6.5 million molecules per library, and more than 10,000 structure-activity relationship (SAR) campaigns run by language-model agents in a single weekend.

The evidence. In a Kaggle competition with about 2,000 teams, Leash released a dataset with billions of physical interactions, a thousand times larger than BindingDB. With shared building blocks, models improved. On truly novel chemistry, nobody beat chance. A separate classifier identified the chemist who made a molecule from the molecule alone with 60% top-5 accuracy across 1,815 chemists. A representation built purely from creator probabilities predicted activity as well as chemical formulas.

The trap. This is the Clever Hans problem. Models trained on public chemistry data learn the sociology of medicinal chemists: what was made, what was skipped, which building blocks the chemist likes. Heavier molecules appear to bind better mostly because chemists keep adding groups to a working lead, not because molecular weight is chemistry. Training on human-generated "exhaust" encodes intent, taste, and institutional history rather than ground-truth physics.

The fix. Leash engineered mutations in binding pockets, rescreened, and used those activity cliffs as hard negatives. Hermes, their deliberately simple cross-attention model, initially ignored binding pockets and memorized short non-conserved protein sequences as a "nickname." Adding mutation data made it spread attention across the real interaction sites. This is a concrete method for testing whether a model learns the problem or a shortcut.

The frontier. The latest shift is to language-model agents with readable English descriptions of molecules and proteins. Agents get 10 binders and 10 non-binders, choose molecules from a catalog, get instant feedback from stored screening data, and update hypotheses. That turns thousands of experiments into reinforcement rollouts. The trade-off: the environment is only as good as the data. DELs miss chemical space, have DNA-barcode artifacts, and still need wet-lab validation before molecules become drugs.

The bottom line. The most transferable lesson for engineering leaders is that data provenance is an architectural decision. If your training data reflects what people chose to build, your model may learn their behavior, not the underlying system. Teams need adversarial eval splits, negative examples from engineered perturbations, and a way to distinguish signal from style.