The Perception-Robustness Tradeoff in Deterministic Image Restoration
This work addresses a fundamental limitation in image restoration for researchers and practitioners, showing that high performance comes at the cost of robustness, which is incremental in highlighting a specific tradeoff.
The paper tackles the tradeoff between perceptual quality and measurement consistency in deterministic image restoration methods, proving that achieving both requires an unbounded Lipschitz constant, making models more vulnerable to adversarial attacks, with demonstrations on super-resolution algorithms.
We study the behavior of deterministic methods for solving inverse problems in imaging. These methods are commonly designed to achieve two goals: (1) attaining high perceptual quality, and (2) generating reconstructions that are consistent with the measurements. We provide a rigorous proof that the better a predictor satisfies these two requirements, the larger its Lipschitz constant must be, regardless of the nature of the degradation involved. In particular, to approach perfect perceptual quality and perfect consistency, the Lipschitz constant of the model must grow to infinity. This implies that such methods are necessarily more susceptible to adversarial attacks. We demonstrate our theory on single image super-resolution algorithms, addressing both noisy and noiseless settings. We also show how this undesired behavior can be leveraged to explore the posterior distribution, thereby allowing the deterministic model to imitate stochastic methods.