Balanced Neural ODEs: nonlinear model order reduction and Koopman operator approximations
This work addresses a problem in machine learning for dynamical systems, offering a flexible and robust method for time series modeling, though it appears incremental as it builds on existing VAEs and Neural ODEs.
The paper tackles the challenge of dimensionality reduction in systems with time-varying inputs by combining Variational Autoencoders and Neural ODEs, resulting in a method called Balanced Neural ODE that enables efficient Koopman operator approximation without predefined dimensionality and demonstrates effectiveness on academic and real-world test cases.
Variational Autoencoders (VAEs) are a powerful framework for learning latent representations of reduced dimensionality, while Neural ODEs excel in learning transient system dynamics. This work combines the strengths of both to generate fast surrogate models with adjustable complexity reacting on time-varying inputs signals. By leveraging the VAE's dimensionality reduction using a nonhierarchical prior, our method adaptively assigns stochastic noise, naturally complementing known NeuralODE training enhancements and enabling probabilistic time series modeling. We show that standard Latent ODEs struggle with dimensionality reduction in systems with time-varying inputs. Our approach mitigates this by continuously propagating variational parameters through time, establishing fixed information channels in latent space. This results in a flexible and robust method that can learn different system complexities, e.g. deep neural networks or linear matrices. Hereby, it enables efficient approximation of the Koopman operator without the need for predefining its dimensionality. As our method balances dimensionality reduction and reconstruction accuracy, we call it Balanced Neural ODE (B-NODE). We demonstrate the effectiveness of this methods on several academic and real-world test cases, e.g. a power plant or MuJoCo data.