Rendering Natural Camera Bokeh Effect with Deep Learning
This addresses the artistic limitation of mobile photography for users seeking professional-looking shallow depth-of-field effects, though it is incremental as it builds on existing deep learning techniques for image enhancement.
The paper tackled the problem of simulating realistic bokeh effects for mobile cameras, which lack the optical capability, by training a deep learning model on a dataset of 5K image pairs from DSLR cameras, resulting in plausible non-uniform bokeh rendering even with complex scenes.
Bokeh is an important artistic effect used to highlight the main object of interest on the photo by blurring all out-of-focus areas. While DSLR and system camera lenses can render this effect naturally, mobile cameras are unable to produce shallow depth-of-field photos due to a very small aperture diameter of their optics. Unlike the current solutions simulating bokeh by applying Gaussian blur to image background, in this paper we propose to learn a realistic shallow focus technique directly from the photos produced by DSLR cameras. For this, we present a large-scale bokeh dataset consisting of 5K shallow / wide depth-of-field image pairs captured using the Canon 7D DSLR with 50mm f/1.8 lenses. We use these images to train a deep learning model to reproduce a natural bokeh effect based on a single narrow-aperture image. The experimental results show that the proposed approach is able to render a plausible non-uniform bokeh even in case of complex input data with multiple objects. The dataset, pre-trained models and codes used in this paper are available on the project website.