Faraway-Frustum: Dealing with Lidar Sparsity for 3D Object Detection using Fusion
This addresses a critical safety issue for fast-moving autonomous vehicles by improving detection at ranges over 60 meters, though it is an incremental improvement over existing fusion methods.
The paper tackles the problem of 3D object detection at long distances where lidar pointclouds become too sparse for reliable detection, proposing a fusion strategy that uses 2D vision for object classification and pointcloud data for localization, which outperforms state-of-the-art methods on the KITTI dataset for faraway objects.
Learned pointcloud representations do not generalize well with an increase in distance to the sensor. For example, at a range greater than 60 meters, the sparsity of lidar pointclouds reaches to a point where even humans cannot discern object shapes from each other. However, this distance should not be considered very far for fast-moving vehicles: A vehicle can traverse 60 meters under two seconds while moving at 70 mph. For safe and robust driving automation, acute 3D object detection at these ranges is indispensable. Against this backdrop, we introduce faraway-frustum: a novel fusion strategy for detecting faraway objects. The main strategy is to depend solely on the 2D vision for recognizing object class, as object shape does not change drastically with an increase in depth, and use pointcloud data for object localization in the 3D space for faraway objects. For closer objects, we use learned pointcloud representations instead, following state-of-the-art. This strategy alleviates the main shortcoming of object detection with learned pointcloud representations. Experiments on the KITTI dataset demonstrate that our method outperforms state-of-the-art by a considerable margin for faraway object detection in bird's-eye-view and 3D. Our code is open-source and publicly available: https://github.com/dongfang-steven-yang/faraway-frustum.