Incremental Search Space Construction for Machine Learning Pipeline Synthesis
This work addresses the challenge of constructing complex ML pipelines in AutoML, which is incremental as it builds on existing methods for algorithm selection and hyperparameter optimization.
The paper tackles the problem of automated machine learning (AutoML) pipeline synthesis by proposing an incremental search space construction method that uses meta-features to prune the search space efficiently, resulting in flexible and dataset-specific pipelines and demonstrating effectiveness on 28 benchmark datasets compared to state-of-the-art frameworks.
Automated machine learning (AutoML) aims for constructing machine learning (ML) pipelines automatically. Many studies have investigated efficient methods for algorithm selection and hyperparameter optimization. However, methods for ML pipeline synthesis and optimization considering the impact of complex pipeline structures containing multiple preprocessing and classification algorithms have not been studied thoroughly. In this paper, we propose a data-centric approach based on meta-features for pipeline construction and hyperparameter optimization inspired by human behavior. By expanding the pipeline search space incrementally in combination with meta-features of intermediate data sets, we are able to prune the pipeline structure search space efficiently. Consequently, flexible and data set specific ML pipelines can be constructed. We prove the effectiveness and competitiveness of our approach on 28 data sets used in well-established AutoML benchmarks in comparison with state-of-the-art AutoML frameworks.