Variational Sampling of Temporal Trajectories
This work addresses a bottleneck in modeling temporal processes for applications like reinforcement learning simulation and evaluation, offering a novel method for trajectory sampling and inference.
The paper tackles the problem of sampling and statistical inference of temporal trajectories, which existing methods struggle with due to parameterization constraints, by introducing a framework that learns the distribution of trajectories through explicit parameterization of the transition function, enabling efficient synthesis of novel trajectories and providing tools for inference such as uncertainty estimation and likelihood evaluations.
A deterministic temporal process can be determined by its trajectory, an element in the product space of (a) initial condition $z_0 \in \mathcal{Z}$ and (b) transition function $f: (\mathcal{Z}, \mathcal{T}) \to \mathcal{Z}$ often influenced by the control of the underlying dynamical system. Existing methods often model the transition function as a differential equation or as a recurrent neural network. Despite their effectiveness in predicting future measurements, few results have successfully established a method for sampling and statistical inference of trajectories using neural networks, partially due to constraints in the parameterization. In this work, we introduce a mechanism to learn the distribution of trajectories by parameterizing the transition function $f$ explicitly as an element in a function space. Our framework allows efficient synthesis of novel trajectories, while also directly providing a convenient tool for inference, i.e., uncertainty estimation, likelihood evaluations and out of distribution detection for abnormal trajectories. These capabilities can have implications for various downstream tasks, e.g., simulation and evaluation for reinforcement learning.