Block-Bench: A Framework for Controllable and Transparent Discrete Optimization Benchmarking
This work provides a tool for researchers in optimization and evolutionary algorithms to create transparent benchmarks, though it is incremental as it builds on existing benchmarking methods.
The authors tackled the problem of constructing discrete optimization benchmarks with fine-grained control over problem properties, resulting in a framework that enables analysis of algorithm behavior in both objective space and variable representations, particularly useful for large-scale multi-modal problems.
We present a novel approach for constructing discrete optimization benchmarks that enables fine-grained control over problem properties, and such benchmarks can facilitate analyzing discrete algorithm behaviors. We build benchmark problems based on a set of block functions, where each block function maps a subset of variables to a real value. Problems are instantiated through a set of block functions, weight factors, and an adjacency graph representing the dependency among the block functions. Through analyzing intermediate block values, our framework allows to analyze algorithm behavior not only in the objective space but also at the level of variable representations in the obtained solutions. This capacity is particularly useful for analyzing discrete heuristics in large-scale multi-modal problems, thereby enhancing the practical relevance of benchmark studies. We demonstrate how the proposed approach can inspire the related work in self-adaptation and diversity control in evolutionary algorithms. Moreover, we explain that the proposed benchmark design enables explicit control over problem properties, supporting research in broader domains such as dynamic algorithm configuration and multi-objective optimization.