Control Variate Score Matching for Diffusion Models
This work addresses a fundamental bottleneck in diffusion model sampling for researchers and practitioners, offering a plug-in solution to enhance efficiency in both training and inference.
The authors tackled the variance trade-off between two score estimators in diffusion models by unifying them with control variates, resulting in a new estimator (CVSI) that theoretically minimizes variance across all noise levels and significantly improves sample efficiency.
Diffusion models offer a robust framework for sampling from unnormalized probability densities, which requires accurately estimating the score of the noise-perturbed target distribution. While the standard Denoising Score Identity (DSI) relies on data samples, access to the target energy function enables an alternative formulation via the Target Score Identity (TSI). However, these estimators face a fundamental variance trade-off: DSI exhibits high variance in low-noise regimes, whereas TSI suffers from high variance at high noise levels. In this work, we reconcile these approaches by unifying both estimators within the principled framework of control variates. We introduce the Control Variate Score Identity (CVSI), deriving an optimal, time-dependent control coefficient that theoretically guarantees variance minimization across the entire noise spectrum. We demonstrate that CVSI serves as a robust, low-variance plug-in estimator that significantly enhances sample efficiency in both data-free sampler learning and inference-time diffusion sampling.