Anytime-Lidar: Deadline-aware 3D Object Detection
This work addresses real-time perception challenges for autonomous systems by enabling deadline-aware 3D object detection, though it is incremental as it builds on existing networks like PointPillars.
The paper tackles the problem of meeting computational deadlines in 3D object detection by introducing a scheduling framework that dynamically selects neural network components to trade off time and accuracy, resulting in significant accuracy improvements under various deadline constraints compared to baselines.
In this work, we present a novel scheduling framework enabling anytime perception for deep neural network (DNN) based 3D object detection pipelines. We focus on computationally expensive region proposal network (RPN) and per-category multi-head detector components, which are common in 3D object detection pipelines, and make them deadline-aware. We propose a scheduling algorithm, which intelligently selects the subset of the components to make effective time and accuracy trade-off on the fly. We minimize accuracy loss of skipping some of the neural network sub-components by projecting previously detected objects onto the current scene through estimations. We apply our approach to a state-of-art 3D object detection network, PointPillars, and evaluate its performance on Jetson Xavier AGX using nuScenes dataset. Compared to the baselines, our approach significantly improve the network's accuracy under various deadline constraints.