Stochastic Prediction of Multi-Agent Interactions from Partial Observations
This work addresses the challenge of forecasting agent behaviors in dynamic environments like sports, which is incremental as it builds on existing graph-based and recurrent neural network methods.
The paper tackles the problem of predicting multi-agent interactions from partial observations by integrating temporal dynamics and ambiguous visual information using a Graph-VRNN, achieving superior performance over baselines on real basketball and synthetic soccer datasets.
We present a method that learns to integrate temporal information, from a learned dynamics model, with ambiguous visual information, from a learned vision model, in the context of interacting agents. Our method is based on a graph-structured variational recurrent neural network (Graph-VRNN), which is trained end-to-end to infer the current state of the (partially observed) world, as well as to forecast future states. We show that our method outperforms various baselines on two sports datasets, one based on real basketball trajectories, and one generated by a soccer game engine.