Expanding Horizons in HCI Research Through LLM-Driven Qualitative Analysis
This addresses qualitative analysis challenges for HCI researchers, presenting an incremental advancement by applying LLMs to this domain.
The paper tackles qualitative analysis in HCI by introducing an LLM-driven method, finding that LLMs match traditional methods' efficacy and offer unique insights, with performance evaluated using SBART cosine similarity.
How would research be like if we still needed to "send" papers typed with a typewriter? Our life and research environment have continually evolved, often accompanied by controversial opinions about new methodologies. In this paper, we embrace this change by introducing a new approach to qualitative analysis in HCI using Large Language Models (LLMs). We detail a method that uses LLMs for qualitative data analysis and present a quantitative framework using SBART cosine similarity for performance evaluation. Our findings indicate that LLMs not only match the efficacy of traditional analysis methods but also offer unique insights. Through a novel dataset and benchmark, we explore LLMs' characteristics in HCI research, suggesting potential avenues for further exploration and application in the field.