Generating Multi-Agent Trajectories using Programmatic Weak Supervision
This work addresses the challenge of modeling complex multi-agent interactions in domains like sports, offering an interpretable approach for generating realistic trajectories, though it is incremental as it extends existing weak supervision methods to spatiotemporal settings.
The paper tackles the problem of training sequential generative models for coordinated multi-agent trajectories, such as in basketball, by introducing a hierarchical framework that uses programmatic weak supervision to capture interpretable high-level behaviors, resulting in realistic trajectory generation validated through quantitative and qualitative evaluations including a user study with professional analysts.
We study the problem of training sequential generative models for capturing coordinated multi-agent trajectory behavior, such as offensive basketball gameplay. When modeling such settings, it is often beneficial to design hierarchical models that can capture long-term coordination using intermediate variables. Furthermore, these intermediate variables should capture interesting high-level behavioral semantics in an interpretable and manipulatable way. We present a hierarchical framework that can effectively learn such sequential generative models. Our approach is inspired by recent work on leveraging programmatically produced weak labels, which we extend to the spatiotemporal regime. In addition to synthetic settings, we show how to instantiate our framework to effectively model complex interactions between basketball players and generate realistic multi-agent trajectories of basketball gameplay over long time periods. We validate our approach using both quantitative and qualitative evaluations, including a user study comparison conducted with professional sports analysts.