Improving Algorithm-Selection and Performance-Prediction via Learning Discriminating Training Samples
This work addresses algorithm selection and performance prediction for solver portfolios, but it is incremental as it builds on existing trajectory-based methods with a tuning enhancement.
The paper tackled the problem of unreliable solver discrimination in feature-free algorithm selection by proposing a meta approach to generate discriminatory trajectories using a tuned Simulated Annealing algorithm, resulting in significantly improved performance metrics for ML models compared to raw trajectory and landscape feature data.
The choice of input-data used to train algorithm-selection models is recognised as being a critical part of the model success. Recently, feature-free methods for algorithm-selection that use short trajectories obtained from running a solver as input have shown promise. However, it is unclear to what extent these trajectories reliably discriminate between solvers. We propose a meta approach to generating discriminatory trajectories with respect to a portfolio of solvers. The algorithm-configuration tool irace is used to tune the parameters of a simple Simulated Annealing algorithm (SA) to produce trajectories that maximise the performance metrics of ML models trained on this data. We show that when the trajectories obtained from the tuned SA algorithm are used in ML models for algorithm-selection and performance prediction, we obtain significantly improved performance metrics compared to models trained both on raw trajectory data and on exploratory landscape features.