On Geometry Regularization in Autoencoder Reduced-Order Models with Latent Neural ODE Dynamics
This work addresses the problem of improving the performance of autoencoder reduced-order models for scientists and engineers working with complex systems like the advection-diffusion-reaction equation, and presents an incremental improvement in the field of reduced-order modeling.
The authors investigated geometric regularization strategies for autoencoder reduced-order models and found that Stiefel projection of the first decoder layer consistently improves conditioning-related diagnostics and rollout performance, while other methods often make subsequent training more difficult. The Stiefel projection method yielded better results, especially for long-horizon rollouts.
We investigate geometric regularization strategies for learned latent representations in encoder--decoder reduced-order models. In a fixed experimental setting for the advection--diffusion--reaction (ADR) equation, we model latent dynamics using a neural ODE and evaluate four regularization approaches applied during autoencoder pre-training: (a) near-isometry regularization of the decoder Jacobian, (b) a stochastic decoder gain penalty based on random directional gains, (c) a second-order directional curvature penalty, and (d) Stiefel projection of the first decoder layer. Across multiple seeds, we find that (a)--(c) often produce latent representations that make subsequent latent-dynamics training with a frozen autoencoder more difficult, especially for long-horizon rollouts, even when they improve local decoder smoothness or related sensitivity proxies. In contrast, (d) consistently improves conditioning-related diagnostics of the learned latent dynamics and tends to yield better rollout performance. We discuss the hypothesis that, in this setting, the downstream impact of latent-geometry mismatch outweighs the benefits of improved decoder smoothness.