Physics-enhanced Gaussian Process Variational Autoencoder
This work addresses the challenge of unsupervised learning of object dynamics in videos for researchers in machine learning and physics, but it is incremental as it builds on existing variational autoencoder methods by adding physical priors.
The paper tackles the problem of learning latent dynamics from video data without ground truth by proposing a physics-enhanced variational autoencoder that incorporates physical prior knowledge as a Gaussian process prior, resulting in improved efficiency and physically correct predictions, as demonstrated in a simulation with an oscillating particle.
Variational autoencoders allow to learn a lower-dimensional latent space based on high-dimensional input/output data. Using video clips as input data, the encoder may be used to describe the movement of an object in the video without ground truth data (unsupervised learning). Even though the object's dynamics is typically based on first principles, this prior knowledge is mostly ignored in the existing literature. Thus, we propose a physics-enhanced variational autoencoder that places a physical-enhanced Gaussian process prior on the latent dynamics to improve the efficiency of the variational autoencoder and to allow physically correct predictions. The physical prior knowledge expressed as linear dynamical system is here reflected by the Green's function and included in the kernel function of the Gaussian process. The benefits of the proposed approach are highlighted in a simulation with an oscillating particle.