E-3DGS: Gaussian Splatting with Exposure and Motion Events
This addresses real-world 3D reconstruction problems for vision applications in dynamic or poorly lit environments, offering a robust and cost-effective solution with incremental hardware and algorithmic improvements.
The paper tackles 3D reconstruction under challenging conditions like motion blur and low illumination by incorporating a transmittance adjustment device to capture both motion and exposure events for event-based 3D Gaussian Splatting. It demonstrates that exposure events improve fine detail reconstruction, outperforming frame-based cameras in low light and overexposure, and achieves faster, higher-quality results than event-based NeRF while being more cost-effective than RGB-event fusion methods.
Achieving 3D reconstruction from images captured under optimal conditions has been extensively studied in the vision and imaging fields. However, in real-world scenarios, challenges such as motion blur and insufficient illumination often limit the performance of standard frame-based cameras in delivering high-quality images. To address these limitations, we incorporate a transmittance adjustment device at the hardware level, enabling event cameras to capture both motion and exposure events for diverse 3D reconstruction scenarios. Motion events (triggered by camera or object movement) are collected in fast-motion scenarios when the device is inactive, while exposure events (generated through controlled camera exposure) are captured during slower motion to reconstruct grayscale images for high-quality training and optimization of event-based 3D Gaussian Splatting (3DGS). Our framework supports three modes: High-Quality Reconstruction using exposure events, Fast Reconstruction relying on motion events, and Balanced Hybrid optimizing with initial exposure events followed by high-speed motion events. On the EventNeRF dataset, we demonstrate that exposure events significantly improve fine detail reconstruction compared to motion events and outperform frame-based cameras under challenging conditions such as low illumination and overexposure. Furthermore, we introduce EME-3D, a real-world 3D dataset with exposure events, motion events, camera calibration parameters, and sparse point clouds. Our method achieves faster and higher-quality reconstruction than event-based NeRF and is more cost-effective than methods combining event and RGB data. E-3DGS sets a new benchmark for event-based 3D reconstruction with robust performance in challenging conditions and lower hardware demands. The source code and dataset will be available at https://github.com/MasterHow/E-3DGS.