Variance-Reduced Diffusion Sampling via Conditional Score Expectation Identity
This work addresses variance reduction in diffusion sampling, which is important for practitioners in generative modeling and Bayesian inverse problems, though it appears incremental as it builds on existing estimators like Tweedie.
The authors tackled the problem of high variance in diffusion sampling by introducing a Conditional Score Expectation identity and deriving a variance-minimizing blended score estimator, which reduced variance and improved sample quality for fixed computational budgets in numerical experiments.
We introduce and prove a \textbf{Conditional Score Expectation (CSE)} identity: an exact relation for the marginal score of affine diffusion processes that links scores across time via a conditional expectation under the forward dynamics. Motivated by this identity, we propose a CSE-based statistical estimator for the score using a Self-Normalized Importance Sampling (SNIS) procedure with prior samples and forward noise. We analyze its relationship to the standard Tweedie estimator, proving anti-correlation for Gaussian targets and establishing the same behavior for general targets in the small time-step regime. Exploiting this structure, we derive a variance-minimizing blended score estimator given by a state--time dependent convex combination of the CSE and Tweedie estimators. Numerical experiments show that this optimal-blending estimator reduces variance and improves sample quality for a fixed computational budget compared to either baseline. We further extend the framework to Bayesian inverse problems via likelihood-informed SNIS weights, and demonstrate improved reconstruction quality and sample diversity on high-dimensional image reconstruction tasks and PDE-governed inverse problems.