Variational Dynamic Mixtures
This addresses the issue of unphysical predictions in multi-modal time series for applications such as trajectory forecasting, though it appears incremental as it builds on existing variational methods.
The paper tackled the problem of mode-averaging in deep probabilistic time series forecasting, where limited inference models lead to unrealistic predictions like taxi trajectories through buildings, by developing variational dynamic mixtures (VDM) to capture multi-modality, resulting in outperformance on highly multi-modal datasets.
Deep probabilistic time series forecasting models have become an integral part of machine learning. While several powerful generative models have been proposed, we provide evidence that their associated inference models are oftentimes too limited and cause the generative model to predict mode-averaged dynamics. Modeaveraging is problematic since many real-world sequences are highly multi-modal, and their averaged dynamics are unphysical (e.g., predicted taxi trajectories might run through buildings on the street map). To better capture multi-modality, we develop variational dynamic mixtures (VDM): a new variational family to infer sequential latent variables. The VDM approximate posterior at each time step is a mixture density network, whose parameters come from propagating multiple samples through a recurrent architecture. This results in an expressive multi-modal posterior approximation. In an empirical study, we show that VDM outperforms competing approaches on highly multi-modal datasets from different domains.