Fast and Robust Likelihood-Guided Diffusion Posterior Sampling with Amortized Variational Inference
This improves the efficiency-flexibility trade-off in diffusion-based inverse problems, which is important for practical applications like image restoration.
The paper tackles the computational cost of likelihood-guided diffusion posterior sampling for inverse problems by amortizing inner optimization problems, achieving faster inference for in-distribution degradations while maintaining robustness to unseen operators.
Zero-shot diffusion posterior sampling offers a flexible framework for inverse problems by accommodating arbitrary degradation operators at test time, but incurs high computational cost due to repeated likelihood-guided updates. In contrast, previous amortized diffusion approaches enable fast inference by replacing likelihood-based sampling with implicit inference models, but at the expense of robustness to unseen degradations. We introduce an amortization strategy for diffusion posterior sampling that preserves explicit likelihood guidance by amortizing the inner optimization problems arising in variational diffusion posterior sampling. This accelerates inference for in-distribution degradations while maintaining robustness to previously unseen operators, thereby improving the trade-off between efficiency and flexibility in diffusion-based inverse problems.