Sampling from the Mean-Field Stationary Distribution
This work addresses the computational complexity of sampling in mean-field systems, which is relevant for machine learning applications like neural network optimization, but it is incremental as it builds on existing methods with conceptual simplifications and technical improvements.
The paper tackles the problem of sampling from the stationary distribution of mean-field SDEs, which is equivalent to minimizing functionals over probability measures with interaction terms, by decoupling approximation via finite-particle systems and sampling using log-concave methods, leading to improved guarantees such as better optimization for two-layer neural networks in the mean-field regime.
We study the complexity of sampling from the stationary distribution of a mean-field SDE, or equivalently, the complexity of minimizing a functional over the space of probability measures which includes an interaction term. Our main insight is to decouple the two key aspects of this problem: (1) approximation of the mean-field SDE via a finite-particle system, via uniform-in-time propagation of chaos, and (2) sampling from the finite-particle stationary distribution, via standard log-concave samplers. Our approach is conceptually simpler and its flexibility allows for incorporating the state-of-the-art for both algorithms and theory. This leads to improved guarantees in numerous settings, including better guarantees for optimizing certain two-layer neural networks in the mean-field regime. A key technical contribution is to establish a new uniform-in-$N$ log-Sobolev inequality for the stationary distribution of the mean-field Langevin dynamics.