Unsupervised Learning of Equivariant Structure from Sequences
This addresses the challenge of unsupervised symmetry learning in sequences, which is incremental as it builds on existing representation theory methods.
The paper tackles the problem of learning equivariant structures from time sequences without supervision by introducing meta-sequential prediction (MSP), which uses stationary properties to train encoder-decoder models for future prediction, resulting in the emergence of hidden disentangled structures as a by-product.
In this study, we present meta-sequential prediction (MSP), an unsupervised framework to learn the symmetry from the time sequence of length at least three. Our method leverages the stationary property (e.g. constant velocity, constant acceleration) of the time sequence to learn the underlying equivariant structure of the dataset by simply training the encoder-decoder model to be able to predict the future observations. We will demonstrate that, with our framework, the hidden disentangled structure of the dataset naturally emerges as a by-product by applying simultaneous block-diagonalization to the transition operators in the latent space, the procedure which is commonly used in representation theory to decompose the feature-space based on the type of response to group actions. We will showcase our method from both empirical and theoretical perspectives. Our result suggests that finding a simple structured relation and learning a model with extrapolation capability are two sides of the same coin. The code is available at https://github.com/takerum/meta_sequential_prediction.