Penalized Langevin dynamics with vanishing penalty for smooth and log-concave targets
This work addresses sampling and optimization challenges in machine learning and statistics, offering incremental improvements in theoretical guarantees for convex settings.
The paper tackles the problem of sampling from smooth, log-concave probability distributions by introducing Penalized Langevin dynamics (PLD) with a vanishing penalty, establishing an upper bound on the Wasserstein-2 distance to the target distribution that depends on the penalty decay rate, and deriving a nonasymptotic convergence guarantee for optimization via the low-temperature limit.
We study the problem of sampling from a probability distribution on $\mathbb R^p$ defined via a convex and smooth potential function. We consider a continuous-time diffusion-type process, termed Penalized Langevin dynamics (PLD), the drift of which is the negative gradient of the potential plus a linear penalty that vanishes when time goes to infinity. An upper bound on the Wasserstein-2 distance between the distribution of the PLD at time $t$ and the target is established. This upper bound highlights the influence of the speed of decay of the penalty on the accuracy of the approximation. As a consequence, considering the low-temperature limit we infer a new nonasymptotic guarantee of convergence of the penalized gradient flow for the optimization problem.