Can AutoML outperform humans? An evaluation on popular OpenML datasets using AutoML Benchmark
This addresses the practical question of AutoML's competitiveness for data scientists, though it is incremental as it builds on existing benchmarks.
The paper evaluated whether AutoML frameworks can outperform human data scientists by testing four AutoML frameworks on 12 OpenML datasets and one real-life dataset, finding that AutoML performed better or equal in 7 out of 12 tasks.
In the last few years, Automated Machine Learning (AutoML) has gained much attention. With that said, the question arises whether AutoML can outperform results achieved by human data scientists. This paper compares four AutoML frameworks on 12 different popular datasets from OpenML; six of them supervised classification tasks and the other six supervised regression ones. Additionally, we consider a real-life dataset from one of our recent projects. The results show that the automated frameworks perform better or equal than the machine learning community in 7 out of 12 OpenML tasks.