Improving Intelligence of Evolutionary Algorithms Using Experience Share and Replay
This provides an incremental improvement for researchers and practitioners needing efficient optimization algorithms for complex functions.
The authors tackled the problem of optimizing high-dimensional continuous functions by proposing PESA, a hybrid algorithm combining Particle Swarm Optimization, Evolution Strategy, and Simulated Annealing with shared replay memory. The result showed superior performance on 12 benchmark functions, with better exploration, faster convergence, and ability to find global optima compared to standalone methods.
We propose PESA, a novel approach combining Particle Swarm Optimisation (PSO), Evolution Strategy (ES), and Simulated Annealing (SA) in a hybrid Algorithm, inspired from reinforcement learning. PESA hybridizes the three algorithms by storing their solutions in a shared replay memory. Next, PESA applies prioritized replay to redistribute data between the three algorithms in frequent form based on their fitness and priority values, which significantly enhances sample diversity and algorithm exploration. Additionally, greedy replay is used implicitly within SA to improve PESA exploitation close to the end of evolution. The validation against 12 high-dimensional continuous benchmark functions shows superior performance by PESA against standalone ES, PSO, and SA, under similar initial starting points, hyperparameters, and number of generations. PESA shows much better exploration behaviour, faster convergence, and ability to find the global optima compared to its standalone counterparts. Given the promising performance, PESA can offer an efficient optimisation option, especially after it goes through additional multiprocessing improvements to handle complex and expensive fitness functions.