Extreme AutoML: Analysis of Classification, Regression, and NLP Performance
This work addresses the need for efficient and accurate automated machine learning tools for practitioners, though it appears incremental as it builds on existing AutoML and Extreme Learning Machine concepts.
The authors tackled the problem of hyperparameter selection in machine learning by benchmarking Extreme AutoML against Google's AutoML, finding that Extreme AutoML achieved significant advantages in accuracy, Jaccard Indices, class variance, and training times across classification, regression, and NLP datasets.
Utilizing machine learning techniques has always required choosing hyperparameters. This is true whether one uses a classical technique such as a KNN or very modern neural networks such as Deep Learning. Though in many applications, hyperparameters are chosen by hand, automated methods have become increasingly more common. These automated methods have become collectively known as automated machine learning, or AutoML. Several automated selection algorithms have shown similar or improved performance over state-of-the-art methods. This breakthrough has led to the development of cloud-based services like Google AutoML, which is based on Deep Learning and is widely considered to be the industry leader in AutoML services. Extreme Learning Machines (ELMs) use a fundamentally different type of neural architecture, producing better results at a significantly discounted computational cost. We benchmark the Extreme AutoML technology against Google's AutoML using several popular classification data sets from the University of California at Irvine's (UCI) repository, and several other data sets, observing significant advantages for Extreme AutoML in accuracy, Jaccard Indices, the variance of Jaccard Indices across classes (i.e. class variance) and training times.