Training a Foundation Model for Materials on a Budget
This work addresses the problem of expensive training for materials modeling, making state-of-the-art methods more accessible to research groups with limited budgets, though it is incremental as it builds on existing designs.
The paper tackles the high computational cost of training foundation models for materials modeling by introducing Nequix, a compact E(3)-equivariant potential that achieves third-best overall ranking on benchmarks while reducing training cost by 20 times and inference speed by two orders of magnitude compared to top models.
Foundation models for materials modeling are advancing quickly, but their training remains expensive, often placing state-of-the-art methods out of reach for many research groups. We introduce Nequix, a compact E(3)-equivariant potential that pairs a simplified NequIP design with modern training practices, including equivariant root-mean-square layer normalization and the Muon optimizer, to retain accuracy while substantially reducing compute requirements. Nequix has 700K parameters and was trained in 100 A100 GPU-hours. On the Matbench-Discovery and MDR Phonon benchmarks, Nequix ranks third overall while requiring a 20 times lower training cost than most other methods, and it delivers two orders of magnitude faster inference speed than the current top-ranked model. We release model weights and fully reproducible codebase at https://github.com/atomicarchitects/nequix.