Virtually increasing the measurement frequency of LIDAR sensor utilizing a single RGB camera
This addresses the need for more frequent monitoring of dynamic objects in autonomous driving, though it is incremental as it builds on existing sensor fusion techniques.
The paper tackles the problem of low LIDAR frame rates in intelligent vehicles by using a mono camera to virtually enhance LIDAR measurements, achieving state-of-the-art accuracy and similarity to real measurements on large public datasets.
The frame rates of most 3D LIDAR sensors used in intelligent vehicles are substantially lower than current cameras installed in the same vehicle. This research suggests using a mono camera to virtually enhance the frame rate of LIDARs, allowing the more frequent monitoring of dynamic objects in the surroundings that move quickly. As a first step, dynamic object candidates are identified and tracked in the camera frames. Following that, the LIDAR measurement points of these items are found by clustering in the frustums of 2D bounding boxes. Projecting these to the camera and tracking them to the next camera frame can be used to create 3D-2D correspondences between different timesteps. These correspondences between the last LIDAR frame and the actual camera frame are used to solve the PnP (Perspective-n-Point) problem. Finally, the estimated transformations are applied to the previously measured points to generate virtual measurements. With the proposed estimation, if the ego movement is known, not just static object position can be determined at timesteps where camera measurement is available, but positions of dynamic objects as well. We achieve state-of-the-art performance on large public datasets in terms of accuracy and similarity to real measurements.