Operator Calculus for Population-Based Optimization: A Mean-Field Convergence Theory
This work provides a unified theoretical framework for analyzing convergence of population-based optimization algorithms, which is a foundational contribution for the optimization community.
The paper introduces an operator calculus that unifies a broad class of population-based optimization methods (e.g., evolution strategies, consensus-based optimization) as compositions of three elementary operators (mutation, selection, recombination) acting on probability measures. Under stability conditions, the composite operator yields a transport-reaction-jump PDE, and a modular Lyapunov principle is established to prove exponential convergence of the mean-field dynamics.
Population-based and distributional optimization methods, from evolution strategies and consensus-based optimization to covariance-matrix adaptation and stochastic gradient methods viewed as distributional dynamics, are widely used for nonconvex or black-box problems, yet their convergence analyses remain fragmented across algorithm-specific techniques. We introduce an operator calculus in which a broad class of such methods, after choosing an appropriate state space and, where necessary, augmenting the state by memory or strategy variables, is described as a composition of three elementary operators (mutation, selection, and recombination) acting on probability measures. Under explicit stability and regularity conditions, the composite operator admits a pre-generator whose continuous-time limit is a transport-reaction-jump (TRJ) PDE that preserves the operator splitting. On this foundation we establish a modular Lyapunov principle. If a state-space Lyapunov function both dissipates under the full generator and controls the relevant search-space gauges, then the state-space Lyapunov functional and the induced search errors decay exponentially. The additive generator structure allows dissipation estimates to be assembled operator by operator, providing a toolkit for certifying convergence of composite mean-field algorithms.