Learning State-Space Models of Dynamic Systems from Arbitrary Data using Joint Embedding Predictive Architectures
This provides a new technique for developing more general control algorithms and estimation techniques, with broad applications in robotics, though it appears incremental as it builds on existing JEPA and neural ODE frameworks.
The paper tackles the problem of learning state-space models from arbitrary observation data by integrating Joint Embedding Predictive Architectures (JEPAs) with neural ODEs, resulting in a method that generates structured latent state-space models, as demonstrated on a simple pendulum system using only image data.
With the advent of Joint Embedding Predictive Architectures (JEPAs), which appear to be more capable than reconstruction-based methods, this paper introduces a novel technique for creating world models using continuous-time dynamic systems from arbitrary observation data. The proposed method integrates sequence embeddings with neural ordinary differential equations (neural ODEs). It employs loss functions that enforce contractive embeddings and Lipschitz constants in state transitions to construct a well-organized latent state space. The approach's effectiveness is demonstrated through the generation of structured latent state-space models for a simple pendulum system using only image data. This opens up a new technique for developing more general control algorithms and estimation techniques with broad applications in robotics.