On Measuring and Controlling the Spectral Bias of the Deep Image Prior
This work addresses issues for researchers and practitioners using the deep image prior in inverse imaging problems, offering incremental improvements to enhance control and efficiency.
The paper tackled the practical limitations of the deep image prior, such as lack of control over the prior and the need for an oracle stopping criterion, by introducing a frequency-band measure to characterize spectral bias and proposing Lipschitz-controlled and Gaussian-controlled layers to prevent performance degradation and accelerate convergence, resulting in favorable outcomes across tasks like denoising and super-resolution without performance degradation.
The deep image prior showed that a randomly initialized network with a suitable architecture can be trained to solve inverse imaging problems by simply optimizing it's parameters to reconstruct a single degraded image. However, it suffers from two practical limitations. First, it remains unclear how to control the prior beyond the choice of the network architecture. Second, training requires an oracle stopping criterion as during the optimization the performance degrades after reaching an optimum value. To address these challenges we introduce a frequency-band correspondence measure to characterize the spectral bias of the deep image prior, where low-frequency image signals are learned faster and better than high-frequency counterparts. Based on our observations, we propose techniques to prevent the eventual performance degradation and accelerate convergence. We introduce a Lipschitz-controlled convolution layer and a Gaussian-controlled upsampling layer as plug-in replacements for layers used in the deep architectures. The experiments show that with these changes the performance does not degrade during optimization, relieving us from the need for an oracle stopping criterion. We further outline a stopping criterion to avoid superfluous computation. Finally, we show that our approach obtains favorable results compared to current approaches across various denoising, deblocking, inpainting, super-resolution and detail enhancement tasks. Code is available at \url{https://github.com/shizenglin/Measure-and-Control-Spectral-Bias}.