Towards a Theory-Guided Benchmarking Suite for Discrete Black-Box Optimization Heuristics: Profiling $(1+λ)$ EA Variants on OneMax and LeadingOnes
This work addresses the gap in benchmarking tools for discrete optimization, benefiting researchers by facilitating theory-empirical reconciliation, though it is incremental as it adapts existing continuous methods.
The authors tackled the lack of a widely accepted benchmarking suite for discrete black-box optimization by adapting the COCO software to pseudo-Boolean problems, enabling fine-grained empirical analysis. They demonstrated this by profiling several (1+λ) EA variants on OneMax and LeadingOnes, leading to a refined analysis of optimization time on LeadingOnes.
Theoretical and empirical research on evolutionary computation methods complement each other by providing two fundamentally different approaches towards a better understanding of black-box optimization heuristics. In discrete optimization, both streams developed rather independently of each other, but we observe today an increasing interest in reconciling these two sub-branches. In continuous optimization, the COCO (COmparing Continuous Optimisers) benchmarking suite has established itself as an important platform that theoreticians and practitioners use to exchange research ideas and questions. No widely accepted equivalent exists in the research domain of discrete black-box optimization. Marking an important step towards filling this gap, we adjust the COCO software to pseudo-Boolean optimization problems, and obtain from this a benchmarking environment that allows a fine-grained empirical analysis of discrete black-box heuristics. In this documentation we demonstrate how this test bed can be used to profile the performance of evolutionary algorithms. More concretely, we study the optimization behavior of several $(1+λ)$ EA variants on the two benchmark problems OneMax and LeadingOnes. This comparison motivates a refined analysis for the optimization time of the $(1+λ)$ EA on LeadingOnes.