Event-Based Dense Reconstruction Pipeline
This addresses the limitation of event cameras for applications requiring dense maps, though it is incremental as it combines existing techniques.
The paper tackles the problem of achieving dense 3D reconstruction from event cameras, which typically produce only semi-dense maps, by proposing a pipeline that uses deep learning to reconstruct intensity images from events, followed by structure from motion and multi-view stereo, resulting in dense reconstruction.
Event cameras are a new type of sensors that are different from traditional cameras. Each pixel is triggered asynchronously by event. The trigger event is the change of the brightness irradiated on the pixel. If the increment or decrement of brightness is higher than a certain threshold, an event is output. Compared with traditional cameras, event cameras have the advantages of high dynamic range and no motion blur. Since events are caused by the apparent motion of intensity edges, the majority of 3D reconstructed maps consist only of scene edges, i.e., semi-dense maps, which is not enough for some applications. In this paper, we propose a pipeline to realize event-based dense reconstruction. First, deep learning is used to reconstruct intensity images from events. And then, structure from motion (SfM) is used to estimate camera intrinsic, extrinsic and sparse point cloud. Finally, multi-view stereo (MVS) is used to complete dense reconstruction.