Beyond SMILES: Evaluating Agentic Systems for Drug Discovery
This work addresses the generalization and practical limitations of agentic systems in drug discovery, highlighting incremental improvements needed for real-world applications.
The paper evaluated six agentic systems for drug discovery across 15 task classes, identifying five capability gaps such as lack of support for protein language models and single-objective optimization, and found that architectural limitations, not knowledge, hinder performance.
Agentic systems for drug discovery have demonstrated autonomous synthesis planning, literature mining, and molecular design. We ask how well they generalize. Evaluating six frameworks against 15 task classes drawn from peptide therapeutics, in vivo pharmacology, and resource-constrained settings, we find five capability gaps: no support for protein language models or peptide-specific prediction, no bridges between in vivo and in silico data, reliance on LLM inference with no pathway to ML training or reinforcement learning, assumptions tied to large-pharma resources, and single-objective optimization that ignores safety-efficacy-stability trade-offs. A paired knowledge-probing experiment suggests the bottleneck is architectural rather than epistemic: four frontier LLMs reason about peptides at levels comparable to small molecules, yet no framework exposes this capability. We propose design requirements and a capability matrix for next-generation frameworks that function as computational partners under realistic constraints.