On the evaluation of (meta-)solver approaches
This work addresses the evaluation challenge for meta-solvers in computational problem-solving, but it is incremental as it synthesizes existing metrics without introducing new methods.
The paper reviews performance metrics for evaluating meta-solvers, which combine multiple solvers to potentially improve performance, by outlining their strengths and weaknesses based on recent works.
Meta-solver approaches exploits a number of individual solvers to potentially build a better solver. To assess the performance of meta-solvers, one can simply adopt the metrics typically used for individual solvers (e.g., runtime or solution quality), or employ more specific evaluation metrics (e.g., by measuring how close the meta-solver gets to its virtual best performance). In this paper, based on some recently published works, we provide an overview of different performance metrics for evaluating (meta-)solvers, by underlying their strengths and weaknesses.