Cinematic Gaussians: Real-Time HDR Radiance Fields with Depth of Field
This work addresses practical challenges in immersive viewing and refocusing for computer graphics and VR applications, representing an incremental improvement over existing radiance field techniques.
The paper tackles the limitations of low dynamic range and pinhole camera assumptions in radiance field reconstructions by introducing a method based on 3D Gaussian Splatting that uses multi-view LDR images to reconstruct an HDR radiance field with depth of field, enabling real-time cinematic rendering and outperforming state-of-the-art methods.
Radiance field methods represent the state of the art in reconstructing complex scenes from multi-view photos. However, these reconstructions often suffer from one or both of the following limitations: First, they typically represent scenes in low dynamic range (LDR), which restricts their use to evenly lit environments and hinders immersive viewing experiences. Secondly, their reliance on a pinhole camera model, assuming all scene elements are in focus in the input images, presents practical challenges and complicates refocusing during novel-view synthesis. Addressing these limitations, we present a lightweight method based on 3D Gaussian Splatting that utilizes multi-view LDR images of a scene with varying exposure times, apertures, and focus distances as input to reconstruct a high-dynamic-range (HDR) radiance field. By incorporating analytical convolutions of Gaussians based on a thin-lens camera model as well as a tonemapping module, our reconstructions enable the rendering of HDR content with flexible refocusing capabilities. We demonstrate that our combined treatment of HDR and depth of field facilitates real-time cinematic rendering, outperforming the state of the art.