More Parameters Than Populations: A Systematic Literature Review of Large Language Models within Survey Research
For survey researchers, this review provides a structured overview of LLM applications and challenges, but it is a work-in-progress with no quantitative results.
This systematic literature review examines how Large Language Models (LLMs) are currently applied across the survey research process, covering pre-data collection, data collection, and post-data collection phases, and discusses opportunities and pitfalls based on existing literature.
[Working Paper] Survey research has a long-standing history of being a human-powered field, but one that embraces various technologies for the collection, processing, and analysis of various behavioral, political, and social outcomes of interest, among others. At the same time, Large Language Models (LLMs) bring new technological challenges and prerequisites in order to fully harness their potential. In this paper, we report work-in-progress on a systematic literature review based on keyword searches from multiple large-scale databases as well as citation networks that assesses how LLMs are currently being applied within the survey research process. We synthesize and organize our findings according to the survey research process to include examples of LLM usage across three broad phases: pre-data collection, data collection, and post-data collection. We discuss selected examples of potential use cases for LLMs as well as its pitfalls based on examples from existing literature. Considering survey research has rich experience and history regarding data quality, we discuss some opportunities and describe future outlooks for survey research to contribute to the continued development and refinement of LLMs.