An Invariant Compiler for Neural ODEs in AI-Accelerated Scientific Simulation
This addresses the challenge of ensuring physically plausible predictions in AI-accelerated scientific simulations across domains, representing a novel design pattern rather than an incremental improvement.
The paper tackles the problem of neural ODEs violating domain invariants like conservation laws in scientific simulations, which leads to physically implausible solutions and compounded errors. It introduces an invariant compiler framework that enforces invariants by construction, guaranteeing trajectories remain on the admissible manifold up to numerical integration error.
Neural ODEs are increasingly used as continuous-time models for scientific and sensor data, but unconstrained neural ODEs can drift and violate domain invariants (e.g., conservation laws), yielding physically implausible solutions. In turn, this can compound error in long-horizon prediction and surrogate simulation. Existing solutions typically aim to enforce invariance by soft penalties or other forms of regularization, which can reduce overall error but do not guarantee that trajectories will not leave the constraint manifold. We introduce the invariant compiler, a framework that enforces invariants by construction: it treats invariants as first-class types and uses an LLM-driven compilation workflow to translate a generic neural ODE specification into a structure-preserving architecture whose trajectories remain on the admissible manifold in continuous time (and up to numerical integration error in practice). This compiler view cleanly separates what must be preserved (scientific structure) from what is learned from data (dynamics within that structure). It provides a systematic design pattern for invariant-respecting neural surrogates across scientific domains.