Orbis: Overcoming Challenges of Long-Horizon Prediction in Driving World Models
This addresses the challenge of improving prediction accuracy for autonomous driving systems in complex scenarios, though it appears incremental as it builds on existing world model approaches with specific design comparisons.
The paper tackles the problem of long-horizon prediction in autonomous driving world models by developing a model that achieves state-of-the-art performance with 469M parameters trained on 280h of video data, particularly excelling in difficult scenarios like turning maneuvers and urban traffic.
Existing world models for autonomous driving struggle with long-horizon generation and generalization to challenging scenarios. In this work, we develop a model using simple design choices, and without additional supervision or sensors, such as maps, depth, or multiple cameras. We show that our model yields state-of-the-art performance, despite having only 469M parameters and being trained on 280h of video data. It particularly stands out in difficult scenarios like turning maneuvers and urban traffic. We test whether discrete token models possibly have advantages over continuous models based on flow matching. To this end, we set up a hybrid tokenizer that is compatible with both approaches and allows for a side-by-side comparison. Our study concludes in favor of the continuous autoregressive model, which is less brittle on individual design choices and more powerful than the model built on discrete tokens. Code, models and qualitative results are publicly available at https://lmb-freiburg.github.io/orbis.github.io/.