Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks
This work addresses the problem of effective model specification for predictive tasks, which is significant for data scientists and domain experts working together in various domains such as health care and public services, particularly for those who are non-technical stakeholders.
The authors tackled the problem of misalignments between model behavior and decision-makers' expectations in temporal predictive models, and their system Tempo resulted in multidisciplinary teams being able to quickly prune infeasible specifications and identify more promising directions to explore. Through three case studies, Tempo demonstrated its effectiveness in facilitating collaboration between data scientists and domain experts.
Temporal predictive models have the potential to improve decisions in health care, public services, and other domains, yet they often fail to effectively support decision-makers. Prior literature shows that many misalignments between model behavior and decision-makers' expectations stem from issues of model specification, namely how, when, and for whom predictions are made. However, model specifications for predictive tasks are highly technical and difficult for non-data-scientist stakeholders to interpret and critique. To address this challenge we developed Tempo, an interactive system that helps data scientists and domain experts collaboratively iterate on model specifications. Using Tempo's simple yet precise temporal query language, data scientists can quickly prototype specifications with greater transparency about pre-processing choices. Moreover, domain experts can assess performance within data subgroups to validate that models behave as expected. Through three case studies, we demonstrate how Tempo helps multidisciplinary teams quickly prune infeasible specifications and identify more promising directions to explore.