Neural Coarse-Graining: Extracting slowly-varying latent degrees of freedom with neural networks
This work addresses the challenge of unsupervised learning of coarse-grained dynamics in time-series data, which is incremental as it builds on existing neural network methods by incorporating a novel loss function for self-determined prediction goals.
The authors tackled the problem of extracting slowly-varying latent degrees of freedom from time-series data by introducing a neural network loss function that allows the network to autonomously identify and focus on predictable features while discarding distracting elements, resulting in a semi-supervised algorithm capable of extracting latent parameters and segmenting time-series sections without labeled data.
We present a loss function for neural networks that encompasses an idea of trivial versus non-trivial predictions, such that the network jointly determines its own prediction goals and learns to satisfy them. This permits the network to choose sub-sets of a problem which are most amenable to its abilities to focus on solving, while discarding 'distracting' elements that interfere with its learning. To do this, the network first transforms the raw data into a higher-level categorical representation, and then trains a predictor from that new time series to its future. To prevent a trivial solution of mapping the signal to zero, we introduce a measure of non-triviality via a contrast between the prediction error of the learned model with a naive model of the overall signal statistics. The transform can learn to discard uninformative and unpredictable components of the signal in favor of the features which are both highly predictive and highly predictable. This creates a coarse-grained model of the time-series dynamics, focusing on predicting the slowly varying latent parameters which control the statistics of the time-series, rather than predicting the fast details directly. The result is a semi-supervised algorithm which is capable of extracting latent parameters, segmenting sections of time-series with differing statistics, and building a higher-level representation of the underlying dynamics from unlabeled data.