DoF-Gaussian: Controllable Depth-of-Field for 3D Gaussian Splatting
This addresses the problem of generating realistic 3D scenes with depth-of-field effects for applications in computer vision and graphics, representing an incremental improvement over existing 3D-GS methods.
The paper tackles the limitation of 3D Gaussian Splatting in handling shallow depth-of-field images by introducing DoF-Gaussian, a method that enables controllable depth-of-field effects and achieves high-quality results as confirmed through extensive experiments.
Recent advances in 3D Gaussian Splatting (3D-GS) have shown remarkable success in representing 3D scenes and generating high-quality, novel views in real-time. However, 3D-GS and its variants assume that input images are captured based on pinhole imaging and are fully in focus. This assumption limits their applicability, as real-world images often feature shallow depth-of-field (DoF). In this paper, we introduce DoF-Gaussian, a controllable depth-of-field method for 3D-GS. We develop a lens-based imaging model based on geometric optics principles to control DoF effects. To ensure accurate scene geometry, we incorporate depth priors adjusted per scene, and we apply defocus-to-focus adaptation to minimize the gap in the circle of confusion. We also introduce a synthetic dataset to assess refocusing capabilities and the model's ability to learn precise lens parameters. Our framework is customizable and supports various interactive applications. Extensive experiments confirm the effectiveness of our method. Our project is available at https://dof-gaussian.github.io.