GEOM-Drugs Revisited: Toward More Chemically Accurate Benchmarks for 3D Molecule Generation
This work addresses chemically inaccurate benchmarking in 3D molecular generation for researchers in computational chemistry and drug discovery, though it is incremental as it focuses on correcting existing methods rather than introducing new generative models.
The authors tackled flaws in evaluation protocols for 3D molecule generation on the GEOM-Drugs dataset, such as incorrect valency definitions and bugs in bond order calculations, by proposing a corrected framework that includes chemically accurate valency tables and a GFN2-xTB-based benchmark, resulting in updated performance metrics and recommendations for future benchmarking.
Deep generative models have shown significant promise in generating valid 3D molecular structures, with the GEOM-Drugs dataset serving as a key benchmark. However, current evaluation protocols suffer from critical flaws, including incorrect valency definitions, bugs in bond order calculations, and reliance on force fields inconsistent with the reference data. In this work, we revisit GEOM-Drugs and propose a corrected evaluation framework: we identify and fix issues in data preprocessing, construct chemically accurate valency tables, and introduce a GFN2-xTB-based geometry and energy benchmark. We retrain and re-evaluate several leading models under this framework, providing updated performance metrics and practical recommendations for future benchmarking. Our results underscore the need for chemically rigorous evaluation practices in 3D molecular generation. Our recommended evaluation methods and GEOM-Drugs processing scripts are available at https://github.com/isayevlab/geom-drugs-3dgen-evaluation.