Benchmark and Survey of Automated Machine Learning Frameworks
This work addresses the problem of high demand for data scientists by enabling domain experts to use AutoML, but it is incremental as it focuses on benchmarking existing frameworks.
The paper provides a survey of current AutoML methods and benchmarks popular AutoML frameworks on 137 datasets from established benchmark suites, evaluating their performance to reduce the need for specialized data scientists.
Machine learning (ML) has become a vital part in many aspects of our daily life. However, building well performing machine learning applications requires highly specialized data scientists and domain experts. Automated machine learning (AutoML) aims to reduce the demand for data scientists by enabling domain experts to build machine learning applications automatically without extensive knowledge of statistics and machine learning. This paper is a combination of a survey on current AutoML methods and a benchmark of popular AutoML frameworks on real data sets. Driven by the selected frameworks for evaluation, we summarize and review important AutoML techniques and methods concerning every step in building an ML pipeline. The selected AutoML frameworks are evaluated on 137 data sets from established AutoML benchmark suits.