SpikeGS: Reconstruct 3D scene via fast-moving bio-inspired sensors
This addresses the challenge of 3D reconstruction in dynamic, real-world scenarios for applications like robotics or autonomous systems, though it is incremental as it builds on 3D Gaussian Splatting with new sensor integration.
The paper tackles the problem of 3D scene reconstruction from fast-moving cameras, which causes blur and limits existing methods like 3D Gaussian Splatting, by proposing SpikeGS, a framework that integrates spike streams from bio-inspired sensors to reconstruct scenes in 1 second, outperforming existing spike-based and deblur methods in experiments.
3D Gaussian Splatting (3DGS) demonstrates unparalleled superior performance in 3D scene reconstruction. However, 3DGS heavily relies on the sharp images. Fulfilling this requirement can be challenging in real-world scenarios especially when the camera moves fast, which severely limits the application of 3DGS. To address these challenges, we proposed Spike Gausian Splatting (SpikeGS), the first framework that integrates the spike streams into 3DGS pipeline to reconstruct 3D scenes via a fast-moving bio-inspired camera. With accumulation rasterization, interval supervision, and a specially designed pipeline, SpikeGS extracts detailed geometry and texture from high temporal resolution but texture lacking spike stream, reconstructs 3D scenes captured in 1 second. Extensive experiments on multiple synthetic and real-world datasets demonstrate the superiority of SpikeGS compared with existing spike-based and deblur 3D scene reconstruction methods. Codes and data will be released soon.