The Algorithm Selection Competitions 2015 and 2017
This addresses the algorithm selection problem for AI researchers, providing benchmarks and insights, but it is incremental as it reviews existing competitions.
The paper reports on the state of the art in algorithm selection based on competitions in 2015 and 2017, showing that performance improved over time but still has room for improvement in some cases.
The algorithm selection problem is to choose the most suitable algorithm for solving a given problem instance. It leverages the complementarity between different approaches that is present in many areas of AI. We report on the state of the art in algorithm selection, as defined by the Algorithm Selection competitions in 2015 and 2017. The results of these competitions show how the state of the art improved over the years. We show that although performance in some cases is very good, there is still room for improvement in other cases. Finally, we provide insights into why some scenarios are hard, and pose challenges to the community on how to advance the current state of the art.