Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference
This provides a theoretical foundation for efficient inference in time series data, though it builds incrementally on prior contrastive learning work.
The paper tackles probabilistic inference in high-dimensional time series by showing that contrastive learning yields Gaussian representations, enabling closed-form solutions for prediction and planning via matrix inversion, validated with simulations up to 46 dimensions.
Given time series data, how can we answer questions like "what will happen in the future?" and "how did we get here?" These sorts of probabilistic inference questions are challenging when observations are high-dimensional. In this paper, we show how these questions can have compact, closed form solutions in terms of learned representations. The key idea is to apply a variant of contrastive learning to time series data. Prior work already shows that the representations learned by contrastive learning encode a probability ratio. By extending prior work to show that the marginal distribution over representations is Gaussian, we can then prove that joint distribution of representations is also Gaussian. Taken together, these results show that representations learned via temporal contrastive learning follow a Gauss-Markov chain, a graphical model where inference (e.g., prediction, planning) over representations corresponds to inverting a low-dimensional matrix. In one special case, inferring intermediate representations will be equivalent to interpolating between the learned representations. We validate our theory using numerical simulations on tasks up to 46-dimensions.