4D-Net for Learned Multi-Modal Alignment
This addresses object detection for autonomous vehicles, with incremental improvements in leveraging motion and image data.
The paper tackles 3D object detection by using 4D information from 3D point clouds and RGB data over time, achieving state-of-the-art performance on the Waymo Open Dataset with improved detection of distant objects.
We present 4D-Net, a 3D object detection approach, which utilizes 3D Point Cloud and RGB sensing information, both in time. We are able to incorporate the 4D information by performing a novel dynamic connection learning across various feature representations and levels of abstraction, as well as by observing geometric constraints. Our approach outperforms the state-of-the-art and strong baselines on the Waymo Open Dataset. 4D-Net is better able to use motion cues and dense image information to detect distant objects more successfully.