Using Large Language Models for Qualitative Analysis can Introduce Serious Bias
This highlights a critical bias risk in using LLMs for social science research, particularly in sensitive contexts like refugee studies, and is incremental in showing that custom models are preferable.
The paper investigates the use of Large Language Models (LLMs) for analyzing qualitative data from open-ended interviews with Rohingya refugees, finding that LLMs introduce non-random biases in annotations, leading to misleading inferences, whereas simpler supervised models trained on high-quality human annotations result in less measurement error and bias.
Large Language Models (LLMs) are quickly becoming ubiquitous, but the implications for social science research are not yet well understood. This paper asks whether LLMs can help us analyse large-N qualitative data from open-ended interviews, with an application to transcripts of interviews with Rohingya refugees in Cox's Bazaar, Bangladesh. We find that a great deal of caution is needed in using LLMs to annotate text as there is a risk of introducing biases that can lead to misleading inferences. We here mean bias in the technical sense, that the errors that LLMs make in annotating interview transcripts are not random with respect to the characteristics of the interview subjects. Training simpler supervised models on high-quality human annotations with flexible coding leads to less measurement error and bias than LLM annotations. Therefore, given that some high quality annotations are necessary in order to asses whether an LLM introduces bias, we argue that it is probably preferable to train a bespoke model on these annotations than it is to use an LLM for annotation.