Implicit Regularization of the Deep Inverse Prior Trained with Inertia
This work addresses the problem of theoretical uncertainty in neural network-based inverse problem solvers for researchers and practitioners, offering incremental improvements in convergence guarantees.
The authors tackled the lack of theoretical guarantees for neural networks in solving inverse problems by providing convergence and recovery guarantees for self-supervised networks trained with inertia, showing optimal accelerated exponential convergence in continuous-time and linear convergence in discrete-time cases.
Solving inverse problems with neural networks benefits from very few theoretical guarantees when it comes to the recovery guarantees. We provide in this work convergence and recovery guarantees for self-supervised neural networks applied to inverse problems, such as Deep Image/Inverse Prior, and trained with inertia featuring both viscous and geometric Hessian-driven dampings. We study both the continuous-time case, i.e., the trajectory of a dynamical system, and the discrete case leading to an inertial algorithm with an adaptive step-size. We show in the continuous-time case that the network can be trained with an optimal accelerated exponential convergence rate compared to the rate obtained with gradient flow. We also show that training a network with our inertial algorithm enjoys similar recovery guarantees though with a less sharp linear convergence rate.