METDrive: Multi-modal End-to-end Autonomous Driving with Temporal Guidance
This work addresses safety improvements for autonomous driving systems, but it is incremental as it builds on existing multi-modal end-to-end approaches.
The paper tackles the problem of improving safety in autonomous driving by enhancing multi-modal end-to-end systems with temporal guidance from ego state features, resulting in a driving score of 70%, route completion of 94%, and infraction score of 0.78 on CARLA benchmarks.
Multi-modal end-to-end autonomous driving has shown promising advancements in recent work. By embedding more modalities into end-to-end networks, the system's understanding of both static and dynamic aspects of the driving environment is enhanced, thereby improving the safety of autonomous driving. In this paper, we introduce METDrive, an end-to-end system that leverages temporal guidance from the embedded time series features of ego states, including rotation angles, steering, throttle signals, and waypoint vectors. The geometric features derived from perception sensor data and the time series features of ego state data jointly guide the waypoint prediction with the proposed temporal guidance loss function. We evaluated METDrive on the CARLA leaderboard benchmarks, achieving a driving score of 70%, a route completion score of 94%, and an infraction score of 0.78.