BokehDiff: Neural Lens Blur with One-Step Diffusion
This work addresses lens blur rendering for computer graphics and photography applications, representing an incremental improvement over existing methods by combining diffusion models with physics-based constraints.
The paper tackles the problem of generating physically accurate lens blur effects in images by introducing BokehDiff, which uses a one-step diffusion approach with a physics-inspired self-attention module to address artifacts from depth estimation errors. The method achieves high-quality results without requiring paired training data, synthesizing photorealistic foregrounds with transparency using diffusion models.
We introduce BokehDiff, a novel lens blur rendering method that achieves physically accurate and visually appealing outcomes, with the help of generative diffusion prior. Previous methods are bounded by the accuracy of depth estimation, generating artifacts in depth discontinuities. Our method employs a physics-inspired self-attention module that aligns with the image formation process, incorporating depth-dependent circle of confusion constraint and self-occlusion effects. We adapt the diffusion model to the one-step inference scheme without introducing additional noise, and achieve results of high quality and fidelity. To address the lack of scalable paired data, we propose to synthesize photorealistic foregrounds with transparency with diffusion models, balancing authenticity and scene diversity.