Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction
This work addresses the challenge of modeling uncertainty and multimodality in future prediction for applications like planning and safety, though it appears incremental as it builds on existing mixture density network limitations.
The paper tackles the problem of multimodal future prediction by proposing a sampling and fitting framework that predicts multiple future samples and groups them into modes, avoiding mode collapse and training instabilities. The approach is shown to trigger good estimates of multimodal distributions on synthetic and real data.
Future prediction is a fundamental principle of intelligence that helps plan actions and avoid possible dangers. As the future is uncertain to a large extent, modeling the uncertainty and multimodality of the future states is of great relevance. Existing approaches are rather limited in this regard and mostly yield a single hypothesis of the future or, at the best, strongly constrained mixture components that suffer from instabilities in training and mode collapse. In this work, we present an approach that involves the prediction of several samples of the future with a winner-takes-all loss and iterative grouping of samples to multiple modes. Moreover, we discuss how to evaluate predicted multimodal distributions, including the common real scenario, where only a single sample from the ground-truth distribution is available for evaluation. We show on synthetic and real data that the proposed approach triggers good estimates of multimodal distributions and avoids mode collapse. Source code is available at $\href{https://github.com/lmb-freiburg/Multimodal-Future-Prediction}{\text{this https URL.}}$