A Distributed Algorithm for Measure-valued Optimization with Additive Objective
This work addresses optimization in probability measure spaces for applications like Langevin sampling and neural network learning, but it appears incremental as it builds on existing ADMM frameworks.
The authors tackled the problem of solving measure-valued optimization problems with additive objectives, which arise in stochastic learning and control, by proposing a distributed nonparametric algorithm based on a two-layer alternating direction method of multipliers (ADMM), generalizing Euclidean consensus to Wasserstein and Sinkhorn consensus ADMM.
We propose a distributed nonparametric algorithm for solving measure-valued optimization problems with additive objectives. Such problems arise in several contexts in stochastic learning and control including Langevin sampling from an unnormalized prior, mean field neural network learning and Wasserstein gradient flows. The proposed algorithm comprises a two-layer alternating direction method of multipliers (ADMM). The outer-layer ADMM generalizes the Euclidean consensus ADMM to the Wasserstein consensus ADMM, and to its entropy-regularized version Sinkhorn consensus ADMM. The inner-layer ADMM turns out to be a specific instance of the standard Euclidean ADMM. The overall algorithm realizes operator splitting for gradient flows in the manifold of probability measures.