Guess & Guide: Gradient-Free Zero-Shot Diffusion Guidance
This work addresses the computational inefficiency of zero-shot diffusion guidance for researchers and practitioners working on Bayesian inverse problems, offering a faster and Pareto optimal solution.
This paper tackles the computational burden of existing zero-shot diffusion guidance methods for Bayesian inverse problems, which rely on surrogate likelihoods requiring vector-Jacobian products. The authors introduce a lightweight likelihood surrogate that eliminates the need for gradient calculations through the denoiser network, leading to dramatically reduced inference costs while achieving the highest results in multiple tasks.
Pretrained diffusion models serve as effective priors for Bayesian inverse problems. These priors enable zero-shot generation by sampling from the conditional distribution, which avoids the need for task-specific retraining. However, a major limitation of existing methods is their reliance on surrogate likelihoods that require vector-Jacobian products at each denoising step, creating a substantial computational burden. To address this, we introduce a lightweight likelihood surrogate that eliminates the need to calculate gradients through the denoiser network. This enables us to handle diverse inverse problems without backpropagation overhead. Experiments confirm that using our method, the inference cost drops dramatically. At the same time, our approach delivers the highest results in multiple tasks. Broadly speaking, we propose the fastest and Pareto optimal method for Bayesian inverse problems.