Interpret-able feedback for AutoML systems
This addresses a key limitation for non-ML experts using AutoML, enabling them to improve models without external help, though it is incremental as it builds on existing AutoML and active learning concepts.
The paper tackles the problem of AutoML systems failing to provide actionable feedback when models underperform, by introducing an interpretable data feedback solution that suggests new data points for labeling to improve accuracy. The result is a 7-8% accuracy improvement in AutoML, outperforming active learning methods in data efficiency while maintaining interpretability.
Automated machine learning (AutoML) systems aim to enable training machine learning (ML) models for non-ML experts. A shortcoming of these systems is that when they fail to produce a model with high accuracy, the user has no path to improve the model other than hiring a data scientist or learning ML -- this defeats the purpose of AutoML and limits its adoption. We introduce an interpretable data feedback solution for AutoML. Our solution suggests new data points for the user to label (without requiring a pool of unlabeled data) to improve the model's accuracy. Our solution analyzes how features influence the prediction among all ML models in an AutoML ensemble, and we suggest more data samples from feature ranges that have high variance in such analysis. Our evaluation shows that our solution can improve the accuracy of AutoML by 7-8% and significantly outperforms popular active learning solutions in data efficiency, all the while providing the added benefit of being interpretable.