Prediction Is Not Physics: Learning and Evaluating Conserved Quantities in Neural Simulators
For researchers in physics-informed ML and neural simulators, this work highlights the failure of standard diffusion models to conserve energy and evaluates methods for learning conserved quantities, though the findings are incremental and system-specific.
The paper shows that diffusion models can achieve low rollout MSE on Hamiltonian trajectories but fail to preserve energy conservation, with energy standard deviation 7500–36000 times larger than ground truth. They investigate whether neural networks can learn conserved quantities, finding that a structured energy model achieves R² ≥ 0.9999 on clean data, while a black-box CDN requires a small alignment loss to avoid collapse, and under 1% noise the CDN can outperform the structured model.
A diffusion model trained on Hamiltonian trajectories can achieve rollout MSE near $10^{-3}$, but the standard deviation of its energy over time is between 7500 and 36000 times larger than the ground-truth energy standard deviation, indicating a failure to preserve conservation laws. This gap motivates our central question of whether neural networks can learn or select globally conserved quantities from physical trajectories. We investigate this across three Hamiltonian systems: projectile motion, pendulum, and spring-mass. We use a structured $T(v)+V(q)$ energy model, a black-box Conservation Discovery Network (CDN), a polynomial CDN, and a conditional diffusion baseline. The structured network reaches $R^2 \geq 0.9999$ against analytical energy on clean data, while the black-box CDN reaches $R^2 \geq 0.996$ when trained with temporal consistency plus a small alignment loss to analytical energy at $t=0$ ($λ_{\mathrm{align}}=0.2$). With $λ_{\mathrm{align}}=0$, CDN Pearson $R^2$ collapses on pendulum and spring-mass ($< 10^{-3}$), showing that temporal consistency alone is not enough to reliably identify the true energy. Under $1\%$ additive Gaussian noise, the CDN outperforms the structured model on the projectile and spring-mass systems, suggesting that the CDN may be more robust to noisy inputs in this setting. However, the polynomial CDN is sensitive to training configuration: it achieves $R^2=0.78$ under a short training schedule on the pendulum system, but reaches $R^2=0.9998$ with more training time and data, regardless of whether noise is added.