Constrained Sampling with Primal-Dual Langevin Monte Carlo
This addresses a limitation in existing methods for constrained sampling, offering a new approach for applications like enforcing fairness in predictions, though it appears incremental as it adapts optimization techniques to a specific sampling context.
The paper tackles the problem of sampling from a probability distribution while satisfying nonlinear statistical constraints, such as in Bayesian inference for fairness or counterfactual analysis, by proposing a primal-dual Langevin Monte Carlo algorithm (PD-LMC) that achieves convergence under standard assumptions like convexity and log-Sobolev inequalities.
This work considers the problem of sampling from a probability distribution known up to a normalization constant while satisfying a set of statistical constraints specified by the expected values of general nonlinear functions. This problem finds applications in, e.g., Bayesian inference, where it can constrain moments to evaluate counterfactual scenarios or enforce desiderata such as prediction fairness. Methods developed to handle support constraints, such as those based on mirror maps, barriers, and penalties, are not suited for this task. This work therefore relies on gradient descent-ascent dynamics in Wasserstein space to put forward a discrete-time primal-dual Langevin Monte Carlo algorithm (PD-LMC) that simultaneously constrains the target distribution and samples from it. We analyze the convergence of PD-LMC under standard assumptions on the target distribution and constraints, namely (strong) convexity and log-Sobolev inequalities. To do so, we bring classical optimization arguments for saddle-point algorithms to the geometry of Wasserstein space. We illustrate the relevance and effectiveness of PD-LMC in several applications.