Are neural scaling laws leading quantum chemistry astray?
This reveals a critical limitation for building reliable quantum chemical models, showing that scaling alone is insufficient, which is an incremental finding for the quantum chemistry and machine learning communities.
The study tested neural scaling laws in quantum chemistry by scaling model capacity and training data, finding that models trained only on stable structures failed to reproduce the H2 energy curve, and even large foundation models could not learn basic Coulomb's law for simple diatomic molecules.
Neural scaling laws are driving the machine learning community toward training ever-larger foundation models across domains, assuring high accuracy and transferable representations for extrapolative tasks. We test this promise in quantum chemistry by scaling model capacity and training data from quantum chemical calculations. As a generalization task, we evaluate the resulting models' predictions of the bond dissociation energy of neutral H$_2$, the simplest possible molecule. We find that, regardless of dataset size or model capacity, models trained only on stable structures fail dramatically to even qualitatively reproduce the H$_2$ energy curve. Only when compressed and stretched geometries are explicitly included in training do the predictions roughly resemble the correct shape. Nonetheless, the largest foundation models trained on the largest and most diverse datasets containing dissociating diatomics exhibit serious failures on simple diatomic molecules. Most strikingly, they cannot reproduce the trivial repulsive energy curve of two bare protons, revealing their failure to learn the basic Coulomb's law involved in electronic structure theory. These results suggest that scaling alone is insufficient for building reliable quantum chemical models.