From particle swarm optimization to consensus based optimization: stochastic modeling and mean-field limit
This work provides a theoretical framework for understanding the dynamics of PSO and its connection to CBO, which is significant for researchers developing and analyzing global optimization algorithms.
This paper develops a continuous stochastic differential equation (SDE) model for Particle Swarm Optimization (PSO) and derives its mean-field limit as a Vlasov-Fokker-Planck-type equation. By introducing an additional SDE for local best positions and regularizing the global best, the authors overcome memory effects and formally derive the mean-field description, clarifying the link to Consensus Based Optimization (CBO) methods in the small inertia limit.
In this paper we consider a continuous description based on stochastic differential equations of the popular particle swarm optimization (PSO) process for solving global optimization problems and derive in the large particle limit the corresponding mean-field approximation based on Vlasov-Fokker-Planck-type equations. The disadvantage of memory effects induced by the need to store the local best position is overcome by the introduction of an additional differential equation describing the evolution of the local best. A regularization process for the global best permits to formally derive the respective mean-field description. Subsequently, in the small inertia limit, we compute the related macroscopic hydrodynamic equations that clarify the link with the recently introduced consensus based optimization (CBO) methods. Several numerical examples illustrate the mean field process, the small inertia limit and the potential of this general class of global optimization methods.