An explicit formulation of the learned noise predictor $ε_θ({\bf x}_t, t)$ via the forward-process noise $ε_{t}$ in denoising diffusion probabilistic models (DDPMs)
This work provides theoretical insight into diffusion models, which are foundational for generative AI, but it is incremental as it builds on existing theory without introducing new methods or applications.
The paper tackles the problem of understanding the learned noise predictor in denoising diffusion probabilistic models (DDPMs) by deriving an explicit formulation in terms of the forward-process noise, and it presents a novel proof of a fundamental equality used in diffusion models, clarifying its theoretical origins.
In denoising diffusion probabilistic models (DDPMs), the learned noise predictor $ ε_θ ( {\bf x}_t , t)$ is trained to approximate the forward-process noise $ε_t$. The equality $\nabla_{{\bf x}_t} \log q({\bf x}_t) = -\frac 1 {\sqrt {1- {\bar α}_t} } ε_θ ( {\bf x}_t , t)$ plays a fundamental role in both theoretical analyses and algorithmic design, and thus is frequently employed across diffusion-based generative models. In this paper, an explicit formulation of $ ε_θ ( {\bf x}_t , t)$ in terms of the forward-process noise $ε_t$ is derived. This result show how the forward-process noise $ε_t$ contributes to the learned predictor $ ε_θ ( {\bf x}_t , t)$. Furthermore, based on this formulation, we present a novel and mathematically rigorous proof of the fundamental equality above, clarifying its origin and providing new theoretical insight into the structure of diffusion models.