Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matching
This work addresses a specific bottleneck in diffusion models for researchers and practitioners, offering an incremental improvement over existing covariance approximation methods.
The paper tackles the problem of inaccurate covariance prediction in diffusion models by introducing Optimal Covariance Matching (OCM), a method that directly regresses optimal diagonal analytic covariances, which significantly reduces approximation error and improves sampling efficiency, recall rate, and likelihood.
The probabilistic diffusion model has become highly effective across various domains. Typically, sampling from a diffusion model involves using a denoising distribution characterized by a Gaussian with a learned mean and either fixed or learned covariances. In this paper, we leverage the recently proposed covariance moment matching technique and introduce a novel method for learning the diagonal covariance. Unlike traditional data-driven diagonal covariance approximation approaches, our method involves directly regressing the optimal diagonal analytic covariance using a new, unbiased objective named Optimal Covariance Matching (OCM). This approach can significantly reduce the approximation error in covariance prediction. We demonstrate how our method can substantially enhance the sampling efficiency, recall rate and likelihood of commonly used diffusion models.