MiLA: Multi-view Intensive-fidelity Long-term Video Generation World Model for Autonomous Driving
This addresses the problem of rare and diverse training data for autonomous driving systems, though it appears incremental as it builds on existing world model approaches.
The paper tackles the challenge of generating long, consistent videos for autonomous driving training by proposing MiLA, a framework that achieves state-of-the-art performance in generating high-fidelity videos up to one minute on the nuScenes dataset.
In recent years, data-driven techniques have greatly advanced autonomous driving systems, but the need for rare and diverse training data remains a challenge, requiring significant investment in equipment and labor. World models, which predict and generate future environmental states, offer a promising solution by synthesizing annotated video data for training. However, existing methods struggle to generate long, consistent videos without accumulating errors, especially in dynamic scenes. To address this, we propose MiLA, a novel framework for generating high-fidelity, long-duration videos up to one minute. MiLA utilizes a Coarse-to-Re(fine) approach to both stabilize video generation and correct distortion of dynamic objects. Additionally, we introduce a Temporal Progressive Denoising Scheduler and Joint Denoising and Correcting Flow modules to improve the quality of generated videos. Extensive experiments on the nuScenes dataset show that MiLA achieves state-of-the-art performance in video generation quality. For more information, visit the project website: https://github.com/xiaomi-mlab/mila.github.io.