Differentiable SVD based on Moore-Penrose Pseudoinverse for Inverse Imaging Problems
This work solves a numerical instability problem for researchers and practitioners using deep unrolling networks in inverse imaging, though it is incremental as it builds on existing SVD methods.
The paper tackles the non-differentiability of singular value decomposition (SVD) in low-rank regularization-based deep unrolling networks for inverse imaging problems, proposing a differentiable SVD using the Moore-Penrose pseudoinverse that effectively addresses numerical instability while maintaining computational precision in tasks like color image compressed sensing and dynamic MRI reconstruction.
Low-rank regularization-based deep unrolling networks have achieved remarkable success in various inverse imaging problems (IIPs). However, the singular value decomposition (SVD) is non-differentiable when duplicated singular values occur, leading to severe numerical instability during training. In this paper, we propose a differentiable SVD based on the Moore-Penrose pseudoinverse to address this issue. To the best of our knowledge, this is the first work to provide a comprehensive analysis of the differentiability of the trivial SVD. Specifically, we show that the non-differentiability of SVD is essentially due to an underdetermined system of linear equations arising in the derivation process. We utilize the Moore-Penrose pseudoinverse to solve the system, thereby proposing a differentiable SVD. A numerical stability analysis in the context of IIPs is provided. Experimental results in color image compressed sensing and dynamic MRI reconstruction show that our proposed differentiable SVD can effectively address the numerical instability issue while ensuring computational precision. Code is available at https://github.com/yhao-z/SVD-inv.