Universality of Gaussian-Mixture Reverse Kernels in Conditional Diffusion
Provides a theoretical foundation for the expressive power of conditional diffusion models, addressing the problem of density approximation for practitioners using such models.
The paper proves that conditional diffusion models with Gaussian-mixture reverse kernels and ReLU-network logits can approximate any regular target distribution arbitrarily well in conditional KL divergence, with the error vanishing as the diffusion horizon increases.
We prove that conditional diffusion models whose reverse kernels are finite Gaussian mixtures with ReLU-network logits can approximate suitably regular target distributions arbitrarily well in context-averaged conditional KL divergence, up to an irreducible terminal mismatch that typically vanishes with increasing diffusion horizon. A path-space decomposition reduces the output error to this mismatch plus per-step reverse-kernel errors; assuming each reverse kernel factors through a finite-dimensional feature map, each step becomes a static conditional density approximation problem, solved by composing Norets' Gaussian-mixture theory with quantitative ReLU bounds. Under exact terminal matching the resulting neural reverse-kernel class is dense in conditional KL.