Perceptual Image Enhancement for Smartphone Real-Time Applications
This addresses the need for efficient image enhancement on smartphones, enabling real-time applications, though it is incremental as it builds on existing deep learning methods with a focus on lightweight design.
The paper tackles the problem of artifacts in smartphone images by proposing LPIENet, a lightweight network for perceptual image enhancement, achieving competitive performance on standard benchmarks with fewer parameters and operations, and demonstrating real-time processing of 2K resolution images in under 1 second on mid-level smartphones.
Recent advances in camera designs and imaging pipelines allow us to capture high-quality images using smartphones. However, due to the small size and lens limitations of the smartphone cameras, we commonly find artifacts or degradation in the processed images. The most common unpleasant effects are noise artifacts, diffraction artifacts, blur, and HDR overexposure. Deep learning methods for image restoration can successfully remove these artifacts. However, most approaches are not suitable for real-time applications on mobile devices due to their heavy computation and memory requirements. In this paper, we propose LPIENet, a lightweight network for perceptual image enhancement, with the focus on deploying it on smartphones. Our experiments show that, with much fewer parameters and operations, our model can deal with the mentioned artifacts and achieve competitive performance compared with state-of-the-art methods on standard benchmarks. Moreover, to prove the efficiency and reliability of our approach, we deployed the model directly on commercial smartphones and evaluated its performance. Our model can process 2K resolution images under 1 second in mid-level commercial smartphones.