A Differentiable Wave Optics Model for End-to-End Computational Imaging System Optimization
This work addresses a critical bottleneck in computational imaging design for robust, high-performance applications, though it is incremental as it builds on existing differentiable optics simulators.
The paper tackles the challenge of accurately modeling wave optics effects like aberration and diffraction in end-to-end optimization of computational imaging systems, showing that neglecting these effects leads to performance degradation, with systems optimized without wave optics suffering when tested with wave optics.
End-to-end optimization, which simultaneously optimizes optics and algorithms, has emerged as a powerful data-driven method for computational imaging system design. This method achieves joint optimization through backpropagation by incorporating differentiable optics simulators to generate measurements and algorithms to extract information from measurements. However, due to high computational costs, it is challenging to model both aberration and diffraction in light transport for end-to-end optimization of compound optics. Therefore, most existing methods compromise physical accuracy by neglecting wave optics effects or off-axis aberrations, which raises concerns about the robustness of the resulting designs. In this paper, we propose a differentiable optics simulator that efficiently models both aberration and diffraction for compound optics. Using the simulator, we conduct end-to-end optimization on scene reconstruction and classification. Experimental results demonstrate that both lenses and algorithms adopt different configurations depending on whether wave optics is modeled. We also show that systems optimized without wave optics suffer from performance degradation when wave optics effects are introduced during testing. These findings underscore the importance of accurate wave optics modeling in optimizing imaging systems for robust, high-performance applications.