Learning to Reduce Defocus Blur by Realistically Modeling Dual-Pixel Data
This work tackles the data scarcity problem for training deep learning models for defocus deblurring, which is a significant hurdle for researchers and developers working with dual-pixel camera systems.
This paper addresses the challenge of limited dual-pixel (DP) data for defocus deblurring by proposing a method to synthetically generate realistic DP images. Leveraging this synthetic data, they introduce a recurrent convolutional network (RCN) architecture that improves deblurring results for both single-frame and multi-frame DP sensor data.
Recent work has shown impressive results on data-driven defocus deblurring using the two-image views available on modern dual-pixel (DP) sensors. One significant challenge in this line of research is access to DP data. Despite many cameras having DP sensors, only a limited number provide access to the low-level DP sensor images. In addition, capturing training data for defocus deblurring involves a time-consuming and tedious setup requiring the camera's aperture to be adjusted. Some cameras with DP sensors (e.g., smartphones) do not have adjustable apertures, further limiting the ability to produce the necessary training data. We address the data capture bottleneck by proposing a procedure to generate realistic DP data synthetically. Our synthesis approach mimics the optical image formation found on DP sensors and can be applied to virtual scenes rendered with standard computer software. Leveraging these realistic synthetic DP images, we introduce a recurrent convolutional network (RCN) architecture that improves deblurring results and is suitable for use with single-frame and multi-frame data (e.g., video) captured by DP sensors. Finally, we show that our synthetic DP data is useful for training DNN models targeting video deblurring applications where access to DP data remains challenging.