Broken neural scaling laws in materials science
This work addresses the challenge of optimizing machine learning models for materials science tasks, where data limitations are critical, but it is incremental in applying existing scaling law frameworks to a specific domain.
The study tackled the problem of predicting the dielectric function of metals in materials science, where data is scarce, by analyzing neural scaling laws with over 200,000 dielectric functions from ab initio calculations, finding broken scaling laws with dataset size and simple power-law saturation with model parameters.
In materials science, data are scarce and expensive to generate, whether computationally or experimentally. Therefore, it is crucial to identify how model performance scales with dataset size and model capacity to distinguish between data- and model-limited regimes. Neural scaling laws provide a framework for quantifying this behavior and guide the design of materials datasets and machine learning architectures. Here, we investigate neural scaling laws for a paradigmatic materials science task: predicting the dielectric function of metals, a high-dimensional response that governs how solids interact with light. Using over 200,000 dielectric functions from high-throughput ab initio calculations, we study two multi-objective graph neural networks trained to predict the frequency-dependent complex interband dielectric function and the Drude frequency. We observe broken neural scaling laws with respect to dataset size, whereas scaling with the number of model parameters follows a simple power law that rapidly saturates.