Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows
This work provides insights for Auto-ML tool designers to improve usability and effectiveness, but it is incremental as it builds on existing automation efforts without introducing new technical methods.
The paper studied how Auto-ML tools are used in practice through a qualitative study with diverse users, finding that full automation is not ideal and instead tools should support a partnership between users and automation to address varying goals like simplicity and reliability.
Efforts to make machine learning more widely accessible have led to a rapid increase in Auto-ML tools that aim to automate the process of training and deploying machine learning. To understand how Auto-ML tools are used in practice today, we performed a qualitative study with participants ranging from novice hobbyists to industry researchers who use Auto-ML tools. We present insights into the benefits and deficiencies of existing tools, as well as the respective roles of the human and automation in ML workflows. Finally, we discuss design implications for the future of Auto-ML tool development. We argue that instead of full automation being the ultimate goal of Auto-ML, designers of these tools should focus on supporting a partnership between the user and the Auto-ML tool. This means that a range of Auto-ML tools will need to be developed to support varying user goals such as simplicity, reproducibility, and reliability.