Partial Rankings of Optimizers
This work addresses the need for more robust benchmarking in optimization, though it appears incremental as it builds on existing depth functions for partial orders.
The authors tackled the problem of benchmarking optimizers by introducing a framework that uses a union-free generic depth function for partial orders to analyze multiple criteria across test functions, resulting in a method that identifies central or outlying rankings and assesses benchmarking suite quality without aggregation shortcomings.
We introduce a framework for benchmarking optimizers according to multiple criteria over various test functions. Based on a recently introduced union-free generic depth function for partial orders/rankings, it fully exploits the ordinal information and allows for incomparability. Our method describes the distribution of all partial orders/rankings, avoiding the notorious shortcomings of aggregation. This permits to identify test functions that produce central or outlying rankings of optimizers and to assess the quality of benchmarking suites.