Latent Representations of Dynamical Systems: When Two is Better Than One
This work addresses a fundamental limitation in machine learning for dynamical systems prediction, potentially impacting fields like physics and AI where irreversible processes are common.
The paper tackles the problem of predicting dynamical systems by showing that using two different latent mappings for present and future is information-theoretically optimal, outperforming single-mapping methods like PCA with significant gains in performance on coupled harmonic oscillators.
A popular approach for predicting the future of dynamical systems involves mapping them into a lower-dimensional "latent space" where prediction is easier. We show that the information-theoretically optimal approach uses different mappings for present and future, in contrast to state-of-the-art machine-learning approaches where both mappings are the same. We illustrate this dichotomy by predicting the time-evolution of coupled harmonic oscillators with dissipation and thermal noise, showing how the optimal 2-mapping method significantly outperforms principal component analysis and all other approaches that use a single latent representation, and discuss the intuitive reason why two representations are better than one. We conjecture that a single latent representation is optimal only for time-reversible processes, not for e.g. text, speech, music or out-of-equilibrium physical systems.