InfGen: Scenario Generation as Next Token Group Prediction
This addresses the need for high-fidelity, dynamic traffic simulation for training and evaluating autonomous driving systems, representing an incremental improvement over static or log-replay methods.
The paper tackles the problem of generating realistic and interactive traffic simulations for autonomous driving by proposing InfGen, a framework that autoregressively outputs agent states and trajectories, enabling infinite scene generation with dynamic agent populations. Experiments show it produces realistic and diverse traffic behaviors, and reinforcement learning policies trained in these scenarios achieve superior robustness and generalization.
Realistic and interactive traffic simulation is essential for training and evaluating autonomous driving systems. However, most existing data-driven simulation methods rely on static initialization or log-replay data, limiting their ability to model dynamic, long-horizon scenarios with evolving agent populations. We propose InfGen, a scenario generation framework that outputs agent states and trajectories in an autoregressive manner. InfGen represents the entire scene as a sequence of tokens, including traffic light signals, agent states, and motion vectors, and uses a transformer model to simulate traffic over time. This design enables InfGen to continuously insert new agents into traffic, supporting infinite scene generation. Experiments demonstrate that InfGen produces realistic, diverse, and adaptive traffic behaviors. Furthermore, reinforcement learning policies trained in InfGen-generated scenarios achieve superior robustness and generalization, validating its utility as a high-fidelity simulation environment for autonomous driving. More information is available at https://metadriverse.github.io/infgen/.