Neural Networks for Predicting Algorithm Runtime Distributions
This work addresses the need for accurate RTD predictions to improve meta-algorithmic procedures such as algorithm selection and portfolios, though it is incremental as it builds on prior machine learning approaches.
The paper tackled the problem of predicting runtime distributions (RTDs) for stochastic algorithms in combinatorial problems like SAT solving and AI planning, by using neural networks to jointly learn RTD parameters, resulting in better predictions than previous methods, even with limited runtime observations.
Many state-of-the-art algorithms for solving hard combinatorial problems in artificial intelligence (AI) include elements of stochasticity that lead to high variations in runtime, even for a fixed problem instance. Knowledge about the resulting runtime distributions (RTDs) of algorithms on given problem instances can be exploited in various meta-algorithmic procedures, such as algorithm selection, portfolios, and randomized restarts. Previous work has shown that machine learning can be used to individually predict mean, median and variance of RTDs. To establish a new state-of-the-art in predicting RTDs, we demonstrate that the parameters of an RTD should be learned jointly and that neural networks can do this well by directly optimizing the likelihood of an RTD given runtime observations. In an empirical study involving five algorithms for SAT solving and AI planning, we show that neural networks predict the true RTDs of unseen instances better than previous methods, and can even do so when only few runtime observations are available per training instance.