Testing the Robustness of AutoML Systems
This work addresses the problem of data quality for AutoML users, but it is incremental as it focuses on evaluating existing systems rather than proposing new methods.
The study tested the robustness of three AutoML systems (TPOT, H2O, AutoKeras) by examining how dirty data affects accuracy and pipeline structure, finding that using dirty training data can help create more robust solutions.
Automated machine learning (AutoML) systems aim at finding the best machine learning (ML) pipeline that automatically matches the task and data at hand. We investigate the robustness of machine learning pipelines generated with three AutoML systems, TPOT, H2O, and AutoKeras. In particular, we study the influence of dirty data on accuracy, and consider how using dirty training data may help create more robust solutions. Furthermore, we also analyze how the structure of the generated pipelines differs in different cases.