More Than "Means to an End": Supporting Reasoning with Transparently Designed AI Data Science Processes
This addresses the challenge for users in high-stakes domains like medicine to effectively use AI tools for open-ended data science tasks, though it is incremental as it builds on existing HCI and AI design principles.
The paper tackles the problem that end-to-end AI data science tools hinder users from evaluating alternatives and reformulating problems in high-stakes domains like medicine, finding that success is driven by designing intermediate artifacts such as readable query languages and concept definitions to support reasoning and refinement.
Generative artificial intelligence (AI) tools can now help people perform complex data science tasks regardless of their expertise. While these tools have great potential to help more people work with data, their end-to-end approach does not support users in evaluating alternative approaches and reformulating problems, both critical to solving open-ended tasks in high-stakes domains. In this paper, we reflect on two AI data science systems designed for the medical setting and how they function as tools for thought. We find that success in these systems was driven by constructing AI workflows around intentionally-designed intermediate artifacts, such as readable query languages, concept definitions, or input-output examples. Despite opaqueness in other parts of the AI process, these intermediates helped users reason about important analytical choices, refine their initial questions, and contribute their unique knowledge. We invite the HCI community to consider when and how intermediate artifacts should be designed to promote effective data science thinking.