Toward Real-World Light Field Super-Resolution
This work addresses the gap in real-world applicability for light field super-resolution, which is important for fields like computational photography and VR, though it is incremental as it builds on existing deep learning approaches.
The authors tackled the problem of poor performance of deep learning methods for light field super-resolution in real-world scenarios by introducing LytroZoom, the first real-world dataset with paired low- and high-resolution light fields, and OFPNet, a network that enhances omni-frequency components, resulting in models that outperform those trained on synthetic datasets and are generalizable to diverse content and devices.
Deep learning has opened up new possibilities for light field super-resolution (SR), but existing methods trained on synthetic datasets with simple degradations (e.g., bicubic downsampling) suffer from poor performance when applied to complex real-world scenarios. To address this problem, we introduce LytroZoom, the first real-world light field SR dataset capturing paired low- and high-resolution light fields of diverse indoor and outdoor scenes using a Lytro ILLUM camera. Additionally, we propose the Omni-Frequency Projection Network (OFPNet), which decomposes the omni-frequency components and iteratively enhances them through frequency projection operations to address spatially variant degradation processes present in all frequency components. Experiments demonstrate that models trained on LytroZoom outperform those trained on synthetic datasets and are generalizable to diverse content and devices. Quantitative and qualitative evaluations verify the superiority of OFPNet. We believe this work will inspire future research in real-world light field SR.