Bridging Spherical Black-Box Optimizers
For researchers in black-box optimization, this work provides a theoretical unification and practical hybrids that improve performance and robustness, though the gains are incremental.
The paper unifies Evolution Strategies, Consensus-Based Optimization, and Optimization via Integration into a common framework, revealing they differ in fitness aggregation and consensus scope. Hybrid methods are introduced that outperform constituent algorithms on BBO benchmarks and locomotion tasks, with explicit control over flat minima preference and multimodal capabilities.
When gradient information is unavailable, black-box optimization (BBO) methods provide a practical alternative. While Evolution Strategies (ES), Consensus-Based Optimization (CBO), Optimization via Integration (OVI), and related methods have each been studied independently, their connections remain underexplored. We unify these approaches within a common theoretical framework, revealing that they differ primarily in two design choices: fitness aggregation (controlling sharpness preference) and consensus scope (controlling modality). Leveraging these insights, we introduce hybrid optimizers that interpolate between existing methods. Our ES-OVI hybrid allows explicit control over the preference for flat minima, enabling a trade-off between performance and robustness in continuous control tasks. Our CBO-OVI hybrids combine the higher-dimensional efficiency of parametric methods with the multimodal capabilities of particle-based approaches, achieving competitive results on language model merging under limited evaluation budgets. We validate our methods on standard BBO benchmarks and higher-dimensional locomotion tasks, demonstrating that the hybrid methods can outperform their constituent algorithms.