Loss Landscape Geometry and the Learning of Symmetries: Or, What Influence Functions Reveal About Robust Generalization
This work addresses the challenge of evaluating symmetry internalization in surrogate models for researchers in scientific machine learning, offering a novel diagnostic technique.
The paper tackles the problem of understanding how neural emulators of PDE solution operators internalize physical symmetries by introducing an influence-based diagnostic that measures gradient coherence along symmetry orbits, showing it provides a mechanism for robust generalization and indicates when training selects symmetry-compatible basins.
We study how neural emulators of partial differential equation solution operators internalize physical symmetries by introducing an influence-based diagnostic that measures the propagation of parameter updates between symmetry-related states, defined as the metric-weighted overlap of loss gradients evaluated along group orbits. This quantity probes the local geometry of the learned loss landscape and goes beyond forward-pass equivariance tests by directly assessing whether learning dynamics couple physically equivalent configurations. Applying our diagnostic to autoregressive fluid flow emulators, we show that orbit-wise gradient coherence provides the mechanism for learning to generalize over symmetry transformations and indicates when training selects a symmetry compatible basin. The result is a novel technique for evaluating if surrogate models have internalized symmetry properties of the known solution operator.