Diffusion-PINN Sampler
This work addresses a key bottleneck in diffusion-based sampling methods for researchers in machine learning and statistics, offering a novel approach with proven convergence guarantees.
The paper tackles the challenge of accurately estimating the drift term in reverse diffusion processes for sampling from unnormalized densities, introducing the Diffusion-PINN Sampler (DPS) which uses physics-informed neural networks to solve the governing PDE of log-densities, achieving improved performance in tasks like identifying mixing proportions in isolated components.
Recent success of diffusion models has inspired a surge of interest in developing sampling techniques using reverse diffusion processes. However, accurately estimating the drift term in the reverse stochastic differential equation (SDE) solely from the unnormalized target density poses significant challenges, hindering existing methods from achieving state-of-the-art performance. In this paper, we introduce the Diffusion-PINN Sampler (DPS), a novel diffusion-based sampling algorithm that estimates the drift term by solving the governing partial differential equation of the log-density of the underlying SDE marginals via physics-informed neural networks (PINN). We prove that the error of log-density approximation can be controlled by the PINN residual loss, enabling us to establish convergence guarantees of DPS. Experiments on a variety of sampling tasks demonstrate the effectiveness of our approach, particularly in accurately identifying mixing proportions when the target contains isolated components.