Depth-aware Blending of Smoothed Images for Bokeh Effect Generation
This addresses the problem of creating bokeh effects for users with limited smartphone camera hardware, though it is incremental as it builds on existing challenges and methods.
The paper tackles generating bokeh effects for images from smartphones with single-rear cameras or lacking good auto-focus hardware by proposing an end-to-end deep learning framework that blends original and smoothed images using monocular depth estimation, achieving second place in the AIM 2019 Bokeh Effect Challenge-Perceptual Track and processing HD images in 0.03 seconds.
Bokeh effect is used in photography to capture images where the closer objects look sharp and every-thing else stays out-of-focus. Bokeh photos are generally captured using Single Lens Reflex cameras using shallow depth-of-field. Most of the modern smartphones can take bokeh images by leveraging dual rear cameras or a good auto-focus hardware. However, for smartphones with single-rear camera without a good auto-focus hardware, we have to rely on software to generate bokeh images. This kind of system is also useful to generate bokeh effect in already captured images. In this paper, an end-to-end deep learning framework is proposed to generate high-quality bokeh effect from images. The original image and different versions of smoothed images are blended to generate Bokeh effect with the help of a monocular depth estimation network. The proposed approach is compared against a saliency detection based baseline and a number of approaches proposed in AIM 2019 Challenge on Bokeh Effect Synthesis. Extensive experiments are shown in order to understand different parts of the proposed algorithm. The network is lightweight and can process an HD image in 0.03 seconds. This approach ranked second in AIM 2019 Bokeh effect challenge-Perceptual Track.