EventSplat: 3D Gaussian Splatting from Moving Event Cameras for Real-time Rendering
This addresses the problem of real-time rendering for applications like robotics or AR/VR, though it is incremental as it builds on prior event-to-video and splatting techniques.
The paper tackles novel view synthesis from fast-moving event cameras by using Gaussian Splatting, achieving higher visual fidelity and better performance than existing event-based NeRF methods while being an order of magnitude faster to render.
We introduce a method for using event camera data in novel view synthesis via Gaussian Splatting. Event cameras offer exceptional temporal resolution and a high dynamic range. Leveraging these capabilities allows us to effectively address the novel view synthesis challenge in the presence of fast camera motion. For initialization of the optimization process, our approach uses prior knowledge encoded in an event-to-video model. We also use spline interpolation for obtaining high quality poses along the event camera trajectory. This enhances the reconstruction quality from fast-moving cameras while overcoming the computational limitations traditionally associated with event-based Neural Radiance Field (NeRF) methods. Our experimental evaluation demonstrates that our results achieve higher visual fidelity and better performance than existing event-based NeRF approaches while being an order of magnitude faster to render.