Automatic Machine Learning Derived from Scholarly Big Data
This addresses the time-consuming and knowledge-intensive process of algorithm selection for practitioners, though it is incremental as it builds on existing word embedding and recommendation techniques.
The paper tackles the challenge of selecting optimal machine learning algorithms for new datasets by introducing Sommelier, an expert system that uses word embeddings from academic publications to recommend algorithms, achieving on average 97.7% of optimal accuracy across 121 datasets and 53 algorithms.
One of the challenging aspects of applying machine learning is the need to identify the algorithms that will perform best for a given dataset. This process can be difficult, time consuming and often requires a great deal of domain knowledge. We present Sommelier, an expert system for recommending the machine learning algorithms that should be applied on a previously unseen dataset. Sommelier is based on word embedding representations of the domain knowledge extracted from a large corpus of academic publications. When presented with a new dataset and its problem description, Sommelier leverages a recommendation model trained on the word embedding representation to provide a ranked list of the most relevant algorithms to be used on the dataset. We demonstrate Sommelier's effectiveness by conducting an extensive evaluation on 121 publicly available datasets and 53 classification algorithms. The top algorithms recommended for each dataset by Sommelier were able to achieve on average 97.7% of the optimal accuracy of all surveyed algorithms.