A Complementarity Analysis of the COCO Benchmark Problems and Artificially Generated Problems
This work addresses the challenge of creating representative benchmark sets for numerical optimization algorithms, though it appears incremental in its analysis approach.
The paper analyzed an artificially generated single-objective continuous problem set compared to the COCO benchmark, using Exploratory Landscape Analysis and Singular Value Decomposition to visualize and correlate problems, aiming to reduce bias in benchmark assessments.
When designing a benchmark problem set, it is important to create a set of benchmark problems that are a good generalization of the set of all possible problems. One possible way of easing this difficult task is by using artificially generated problems. In this paper, one such single-objective continuous problem generation approach is analyzed and compared with the COCO benchmark problem set, a well know problem set for benchmarking numerical optimization algorithms. Using Exploratory Landscape Analysis and Singular Value Decomposition, we show that such representations allow us to further explore the relations between the problems by applying visualization and correlation analysis techniques, with the goal of decreasing the bias in benchmark problem assessment.