Evaluating Real-World Generalizability of Algorithm Selection Models
For practitioners applying Algorithm Selection to real-world optimization, this work identifies key limitations in current models' cross-domain generalization.
This paper evaluates the generalization ability of Algorithm Selection models across synthetic benchmarks (BBOB, CEC) and real-world problems (robotics, UAV path-planning), finding that transfer between domains often fails, highlighting challenges for real-world application.
Algorithm Selection (AS) aims to automatically identify the most suitable optimization algorithm for a given problem instance by leveraging measurable problem characteristics and historical performance data. In this study, we investigate the generalization ability of AS models across both synthetic and real-world optimization landscapes. We consider two widely used academic benchmark suites (BBOB and CEC) and two real-world problem sets (robotics trajectory optimization tasks and unmanned aerial vehicle path-planning problems). Through a systematic cross-benchmark evaluation, we analyze how AS models transfer between domains, identify where generalization succeeds or breaks down, and highlight the challenges that arise when applying AS in realistic, domain-specific contexts. Our findings provide insights into the robustness of current AS approaches and inform the development of more reliable, broadly applicable AS systems for real-world optimization.