Degradation-Modeled Multipath Diffusion for Tunable Metalens Photography
This work addresses the problem of computational image restoration for ultra-compact metalens photography, offering a tunable solution that balances fidelity and perceptual quality, though it is incremental as it builds on existing diffusion models and domain-specific techniques.
The paper tackles the challenge of reconstructing high-quality images from metalens-based computational imaging systems, which suffer from optical degradation and restoration difficulties, by introducing a diffusion-based framework that achieves high-fidelity and sharp image reconstruction without requiring large paired datasets.
Metalenses offer significant potential for ultra-compact computational imaging but face challenges from complex optical degradation and computational restoration difficulties. Existing methods typically rely on precise optical calibration or massive paired datasets, which are non-trivial for real-world imaging systems. Furthermore, a lack of control over the inference process often results in undesirable hallucinated artifacts. We introduce Degradation-Modeled Multipath Diffusion for tunable metalens photography, leveraging powerful natural image priors from pretrained models instead of large datasets. Our framework uses positive, neutral, and negative-prompt paths to balance high-frequency detail generation, structural fidelity, and suppression of metalens-specific degradation, alongside \textit{pseudo} data augmentation. A tunable decoder enables controlled trade-offs between fidelity and perceptual quality. Additionally, a spatially varying degradation-aware attention (SVDA) module adaptively models complex optical and sensor-induced degradation. Finally, we design and build a millimeter-scale MetaCamera for real-world validation. Extensive results show that our approach outperforms state-of-the-art methods, achieving high-fidelity and sharp image reconstruction. More materials: https://dmdiff.github.io/.