Sampling with Mollified Interaction Energy Descent
This addresses a fundamental problem in computational statistics and machine learning for researchers and practitioners, offering incremental improvements in constrained sampling scenarios.
The paper tackles the problem of sampling from a target measure with an unknown normalization constant by introducing mollified interaction energy descent (MIED), a new optimization-based method that minimizes mollified interaction energies. The result is a first-order particle-based algorithm that performs competitively with existing methods like SVGD for unconstrained sampling and handles constrained domains more flexibly with strong performance.
Sampling from a target measure whose density is only known up to a normalization constant is a fundamental problem in computational statistics and machine learning. In this paper, we present a new optimization-based method for sampling called mollified interaction energy descent (MIED). MIED minimizes a new class of energies on probability measures called mollified interaction energies (MIEs). These energies rely on mollifier functions -- smooth approximations of the Dirac delta originated from PDE theory. We show that as the mollifier approaches the Dirac delta, the MIE converges to the chi-square divergence with respect to the target measure and the gradient flow of the MIE agrees with that of the chi-square divergence. Optimizing this energy with proper discretization yields a practical first-order particle-based algorithm for sampling in both unconstrained and constrained domains. We show experimentally that for unconstrained sampling problems our algorithm performs on par with existing particle-based algorithms like SVGD, while for constrained sampling problems our method readily incorporates constrained optimization techniques to handle more flexible constraints with strong performance compared to alternatives.