The Devil is in the Upsampling: Architectural Decisions Made Simpler for Denoising with Deep Image Prior
This work addresses practical issues in denoising for computer vision applications, offering a more efficient and robust method, though it is incremental as it builds on the existing Deep Image Prior framework.
The paper tackled the challenge of architectural design and noise-fitting in Deep Image Prior for image denoising by identifying unlearnt upsampling as the key factor, leading to strategies that estimate suitable architectures per image without extensive search, resulting in denoising with up to 95% fewer parameters and better detail preservation.
Deep Image Prior (DIP) shows that some network architectures naturally bias towards smooth images and resist noises, a phenomenon known as spectral bias. Image denoising is an immediate application of this property. Although DIP has removed the requirement of large training sets, it still presents two practical challenges for denoising: architectural design and noise-fitting, which are often intertwined. Existing methods mostly handcraft or search for the architecture from a large design space, due to the lack of understanding on how the architectural choice corresponds to the image. In this study, we analyze from a frequency perspective to demonstrate that the unlearnt upsampling is the main driving force behind the denoising phenomenon in DIP. This finding then leads to strategies for estimating a suitable architecture for every image without a laborious search. Extensive experiments show that the estimated architectures denoise and preserve the textural details better than current methods with up to 95% fewer parameters. The under-parameterized nature also makes them especially robust to a higher level of noise.