Tweedie's Formulae and Diffusion Generative Models Beyond Gaussian
For researchers in generative modeling, this paper provides a theoretical and practical extension of diffusion models to non-Gaussian processes, though the experimental gains are modest and the approach is incremental.
This work extends Tweedie's formula to non-Gaussian processes (GBM, BESQ, CIR) for diffusion generative models, enabling denoising score matching beyond Gaussian noise. Experiments on image and financial time series generation show the potential of these non-Gaussian models.
Diffusion models have achieved remarkable success in generating samples from unknown data distributions. Most popular stochastic differential equation-based diffusion models perturb the target distribution by adding Gaussian noise, transforming it into a simple prior, and then use denoising score matching, a consequence of Tweedie's formula, to learn the score function and generate clean samples from noise. However, non-Gaussian diffusion models with state-dependent diffusion coefficient have been largely underexplored, as have the corresponding Tweedie's formulae. In this work, we extend Tweedie's formula to important non-Gaussian processes, including geometric Brownian motion (GBM), squared Bessel (BESQ) processes, and Cox-Ingersoll-Ross (CIR) processes, thereby yielding the corresponding denoising score-matching objectives. We then apply the derived formulae to image and financial time series generation using GBM- and CIR-based diffusion models, and to empirical Bayes estimation under the BESQ setting. The reported experimental results demonstrate the potential of non-Gaussian models.