Designing for the Long Tail of Machine Learning
This work addresses a problem for UX practitioners by providing a framework to navigate ML design trade-offs, but it is incremental as it builds on existing learning curve theory.
The paper tackles the challenge of integrating machine learning into user-centered design processes by proposing that the theoretical scaling of ML performance with training data can guide trade-offs between data gathering, model development, and interaction design, exemplified through a bootstrap phase pattern.
Recent technical advances has made machine learning (ML) a promising component to include in end user facing systems. However, user experience (UX) practitioners face challenges in relating ML to existing user-centered design processes and how to navigate the possibilities and constraints of this design space. Drawing on our own experience, we characterize designing within this space as navigating trade-offs between data gathering, model development and designing valuable interactions for a given model performance. We suggest that the theoretical description of how machine learning performance scales with training data can guide designers in these trade-offs as well as having implications for prototyping. We exemplify the learning curve's usage by arguing that a useful pattern is to design an initial system in a bootstrap phase that aims to exploit the training effect of data collected at increasing orders of magnitude.