Tune: A Research Platform for Distributed Model Selection and Training
This addresses the need for efficient distributed model selection and training in machine learning, though it is incremental as it builds on existing hyperparameter search methods.
The paper tackles the problem of inefficient adaptation of hyperparameter search algorithms to distributed compute environments by proposing Tune, a unified framework that provides a narrow-waist interface between training scripts and search algorithms, enabling straightforward scaling to large clusters and simplifying algorithm implementation.
Modern machine learning algorithms are increasingly computationally demanding, requiring specialized hardware and distributed computation to achieve high performance in a reasonable time frame. Many hyperparameter search algorithms have been proposed for improving the efficiency of model selection, however their adaptation to the distributed compute environment is often ad-hoc. We propose Tune, a unified framework for model selection and training that provides a narrow-waist interface between training scripts and search algorithms. We show that this interface meets the requirements for a broad range of hyperparameter search algorithms, allows straightforward scaling of search to large clusters, and simplifies algorithm implementation. We demonstrate the implementation of several state-of-the-art hyperparameter search algorithms in Tune. Tune is available at http://ray.readthedocs.io/en/latest/tune.html.