ReasonNet: End-to-End Driving with Temporal and Global Reasoning
This addresses the problem of improving safety and reliability in dense urban traffic for autonomous vehicles, representing an incremental advancement with a novel method for known bottlenecks.
The paper tackles the challenge of predicting scene evolution and handling rare adverse events like occluded objects in autonomous driving by introducing ReasonNet, an end-to-end framework that leverages temporal and global reasoning, which outperforms prior methods on CARLA benchmarks, ranking first on the sensor track.
The large-scale deployment of autonomous vehicles is yet to come, and one of the major remaining challenges lies in urban dense traffic scenarios. In such cases, it remains challenging to predict the future evolution of the scene and future behaviors of objects, and to deal with rare adverse events such as the sudden appearance of occluded objects. In this paper, we present ReasonNet, a novel end-to-end driving framework that extensively exploits both temporal and global information of the driving scene. By reasoning on the temporal behavior of objects, our method can effectively process the interactions and relationships among features in different frames. Reasoning about the global information of the scene can also improve overall perception performance and benefit the detection of adverse events, especially the anticipation of potential danger from occluded objects. For comprehensive evaluation on occlusion events, we also release publicly a driving simulation benchmark DriveOcclusionSim consisting of diverse occlusion events. We conduct extensive experiments on multiple CARLA benchmarks, where our model outperforms all prior methods, ranking first on the sensor track of the public CARLA Leaderboard.