JointDiff: Bridging Continuous and Discrete in Multi-Agent Trajectory Generation
This work addresses the challenge of building realistic and controllable generative models for interactive systems like sports, though it is incremental as it builds on existing diffusion methods.
The authors tackled the problem of separately modeling continuous and discrete processes in generative models by introducing JointDiff, a diffusion framework that simultaneously generates continuous spatio-temporal data and synchronous discrete events, achieving state-of-the-art performance in sports trajectory generation.
Generative models often treat continuous data and discrete events as separate processes, creating a gap in modeling complex systems where they interact synchronously. To bridge this gap, we introduce JointDiff, a novel diffusion framework designed to unify these two processes by simultaneously generating continuous spatio-temporal data and synchronous discrete events. We demonstrate its efficacy in the sports domain by simultaneously modeling multi-agent trajectories and key possession events. This joint modeling is validated with non-controllable generation and two novel controllable generation scenarios: weak-possessor-guidance, which offers flexible semantic control over game dynamics through a simple list of intended ball possessors, and text-guidance, which enables fine-grained, language-driven generation. To enable the conditioning with these guidance signals, we introduce CrossGuid, an effective conditioning operation for multi-agent domains. We also share a new unified sports benchmark enhanced with textual descriptions for soccer and football datasets. JointDiff achieves state-of-the-art performance, demonstrating that joint modeling is crucial for building realistic and controllable generative models for interactive systems.