Progressive Sampling-Based Bayesian Optimization for Efficient and Automatic Machine Learning Model Selection
This addresses the challenge of making machine learning more accessible for clinical data analysis by automating model selection, though it appears incremental as it builds on existing Bayesian optimization methods.
The paper tackles the inefficiency of existing automatic machine learning model selection methods on large datasets by introducing progressive sampling-based Bayesian optimization, which significantly reduces search time, classification error rate, and standard deviation of error rate compared to a state-of-the-art method.
Purpose: Machine learning is broadly used for clinical data analysis. Before training a model, a machine learning algorithm must be selected. Also, the values of one or more model parameters termed hyper-parameters must be set. Selecting algorithms and hyper-parameter values requires advanced machine learning knowledge and many labor-intensive manual iterations. To lower the bar to machine learning, miscellaneous automatic selection methods for algorithms and/or hyper-parameter values have been proposed. Existing automatic selection methods are inefficient on large data sets. This poses a challenge for using machine learning in the clinical big data era. Methods: To address the challenge, this paper presents progressive sampling-based Bayesian optimization, an efficient and automatic selection method for both algorithms and hyper-parameter values. Results: We report an implementation of the method. We show that compared to a state of the art automatic selection method, our method can significantly reduce search time, classification error rate, and standard deviation of error rate due to randomization. Conclusions: This is major progress towards enabling fast turnaround in identifying high-quality solutions required by many machine learning-based clinical data analysis tasks.