Sketched RT3D: How to reconstruct billions of photons per second
This work addresses the computational bottleneck for real-time deployment of lidar arrays, though it is incremental as it modifies an existing method.
The paper tackles the challenge of real-time 3D reconstruction from billions of photons per second in lidar systems by proposing a sketched version of an existing algorithm, achieving the same reconstruction performance while significantly reducing execution time and memory requirements.
Single-photon light detection and ranging (lidar) captures depth and intensity information of a 3D scene. Reconstructing a scene from observed photons is a challenging task due to spurious detections associated with background illumination sources. To tackle this problem, there is a plethora of 3D reconstruction algorithms which exploit spatial regularity of natural scenes to provide stable reconstructions. However, most existing algorithms have computational and memory complexity proportional to the number of recorded photons. This complexity hinders their real-time deployment on modern lidar arrays which acquire billions of photons per second. Leveraging a recent lidar sketching framework, we show that it is possible to modify existing reconstruction algorithms such that they only require a small sketch of the photon information. In particular, we propose a sketched version of a recent state-of-the-art algorithm which uses point cloud denoisers to provide spatially regularized reconstructions. A series of experiments performed on real lidar datasets demonstrates a significant reduction of execution time and memory requirements, while achieving the same reconstruction performance than in the full data case.