PUCA: Patch-Unshuffle and Channel Attention for Enhanced Self-Supervised Image Denoising
This addresses the challenge of real-world image denoising without clean-noisy pairs, offering a solution for applications in photography and medical imaging, though it is incremental as it builds on existing blind-spot networks.
The paper tackles the problem of self-supervised image denoising by proposing PUCA, a novel J-invariant U-Net architecture that uses patch-unshuffle/shuffle and dilated attention blocks to enhance receptive fields and incorporate global context, achieving state-of-the-art performance.
Although supervised image denoising networks have shown remarkable performance on synthesized noisy images, they often fail in practice due to the difference between real and synthesized noise. Since clean-noisy image pairs from the real world are extremely costly to gather, self-supervised learning, which utilizes noisy input itself as a target, has been studied. To prevent a self-supervised denoising model from learning identical mapping, each output pixel should not be influenced by its corresponding input pixel; This requirement is known as J-invariance. Blind-spot networks (BSNs) have been a prevalent choice to ensure J-invariance in self-supervised image denoising. However, constructing variations of BSNs by injecting additional operations such as downsampling can expose blinded information, thereby violating J-invariance. Consequently, convolutions designed specifically for BSNs have been allowed only, limiting architectural flexibility. To overcome this limitation, we propose PUCA, a novel J-invariant U-Net architecture, for self-supervised denoising. PUCA leverages patch-unshuffle/shuffle to dramatically expand receptive fields while maintaining J-invariance and dilated attention blocks (DABs) for global context incorporation. Experimental results demonstrate that PUCA achieves state-of-the-art performance, outperforming existing methods in self-supervised image denoising.