Supporting Serendipity: Opportunities and Challenges for Human-AI Collaboration in Qualitative Analysis
This study addresses the problem of scaling qualitative analysis for CSCW and HCI researchers, providing insights into designing AI tools that respect human agency and the inductive nature of their work.
This paper investigates the challenges qualitative researchers face when analyzing large datasets and explores the potential for human-AI collaboration. The study found that while researchers desire AI assistance for large corpora, they prioritize human agency and serendipity, indicating a need for AI tools that support rather than interfere with their inductive analysis.
Qualitative inductive methods are widely used in CSCW and HCI research for their ability to generatively discover deep and contextualized insights, but these inherently manual and human-resource-intensive processes are often infeasible for analyzing large corpora. Researchers have been increasingly interested in ways to apply qualitative methods to "big" data problems, hoping to achieve more generalizable results from larger amounts of data while preserving the depth and richness of qualitative methods. In this paper, we describe a study of qualitative researchers' work practices and their challenges, with an eye towards whether this is an appropriate domain for human-AI collaboration and what successful collaborations might entail. Our findings characterize participants' diverse methodological practices and nuanced collaboration dynamics, and identify areas where they might benefit from AI-based tools. While participants highlight the messiness and uncertainty of qualitative inductive analysis, they still want full agency over the process and believe that AI should not interfere. Our study provides a deep investigation of task delegability in human-AI collaboration in the context of qualitative analysis, and offers directions for the design of AI assistance that honor serendipity, human agency, and ambiguity.