Mirror Mean-Field Langevin Dynamics
This work addresses constrained optimization in interacting particle systems, such as neural networks, but is incremental as it extends existing methods to handle constraints.
The paper tackles the problem of optimizing probability measures constrained to convex domains, which existing mean-field algorithms cannot handle due to global diffusion terms, by proposing mirror mean-field Langevin dynamics (MMFLD) and achieves linear convergence guarantees and uniform-in-time propagation of chaos results.
The mean-field Langevin dynamics (MFLD) minimizes an entropy-regularized nonlinear convex functional on the Wasserstein space over $\mathbb{R}^d$, and has gained attention recently as a model for the gradient descent dynamics of interacting particle systems such as infinite-width two-layer neural networks. However, many problems of interest have constrained domains, which are not solved by existing mean-field algorithms due to the global diffusion term. We study the optimization of probability measures constrained to a convex subset of $\mathbb{R}^d$ by proposing the \emph{mirror mean-field Langevin dynamics} (MMFLD), an extension of MFLD to the mirror Langevin framework. We obtain linear convergence guarantees for the continuous MMFLD via a uniform log-Sobolev inequality, and uniform-in-time propagation of chaos results for its time- and particle-discretized counterpart.