Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
This work provides a real-time, holistic solution for autonomous driving perception and prediction, improving robustness to occlusion and sparse data for self-driving vehicles.
This paper introduces a deep neural network that jointly performs 3D detection, tracking, and motion forecasting from 3D sensor data. The approach achieves state-of-the-art performance with a large margin on a new large-scale dataset, completing all tasks in 30 ms.
In this paper we propose a novel deep neural network that is able to jointly reason about 3D detection, tracking and motion forecasting given data captured by a 3D sensor. By jointly reasoning about these tasks, our holistic approach is more robust to occlusion as well as sparse data at range. Our approach performs 3D convolutions across space and time over a bird's eye view representation of the 3D world, which is very efficient in terms of both memory and computation. Our experiments on a new very large scale dataset captured in several north american cities, show that we can outperform the state-of-the-art by a large margin. Importantly, by sharing computation we can perform all tasks in as little as 30 ms.