Algorithm Configuration: Learning policies for the quick termination of poor performers
This work addresses the efficiency of algorithm configuration for researchers and practitioners, but it is incremental as it builds on existing termination strategies.
The paper tackled the problem of speeding up algorithm configuration by learning policies to terminate poor-performing configurations early, showing that performance differences between short and long tests vary across domains and proposing a method that adapts to these differences.
One way to speed up the algorithm configuration task is to use short runs instead of long runs as much as possible, but without discarding the configurations that eventually do well on the long runs. We consider the problem of selecting the top performing configurations of the Conditional Markov Chain Search (CMCS), a general algorithm schema that includes, for examples, VNS. We investigate how the structure of performance on short tests links with those on long tests, showing that significant differences arise between test domains. We propose a "performance envelope" method to exploit the links; that learns when runs should be terminated, but that automatically adapts to the domain.