A Proximal Algorithm for Sampling
This addresses a problem in computational statistics and machine learning for researchers dealing with non-smooth sampling tasks, though it appears incremental as it builds on known frameworks.
The paper tackles sampling from potentials that lack smoothness, including convex and non-convex cases, by developing a proximal sampling algorithm based on the alternating sampling framework. The result shows that in almost all cases, this algorithm achieves better complexity than existing methods.
We study sampling problems associated with potentials that lack smoothness. The potentials can be either convex or non-convex. Departing from the standard smooth setting, the potentials are only assumed to be weakly smooth or non-smooth, or the summation of multiple such functions. We develop a sampling algorithm that resembles proximal algorithms in optimization for this challenging sampling task. Our algorithm is based on a special case of Gibbs sampling known as the alternating sampling framework (ASF). The key contribution of this work is a practical realization of the ASF based on rejection sampling for both non-convex and convex potentials that are not necessarily smooth. In almost all the cases of sampling considered in this work, our proximal sampling algorithm achieves better complexity than all existing methods.