Using Zero-Shot LLM-Generated Survey Data for Geographically Explicit Population Synthesis
For researchers needing synthetic populations, this work provides an initial assessment of LLM-generated data as a cost-effective alternative, though results are mixed and incremental.
The paper evaluates whether zero-shot LLM-generated health survey data can replace real survey data in iterative proportional fitting for population synthesis. Results show LLMs capture major state-level contrasts but performance is variable-dependent, with IPF sometimes amplifying errors, indicating promise as supplementary input but not as a replacement.
There is a growing interest in utilizing synthetic populations for a diverse range of applications. At the same time, we are witnessing a tremendous growth in artificial intelligence in all walks of life. This paper evaluates whether zero-shot large language model (LLM)-generated health survey data can serve as inputs to a conventional iterative proportional fitting (IPF) workflow for geographically explicit population synthesis. Using the 2023 Behavioral Risk Factor Surveillance System (BRFSS), we generate synthetic survey records for the U.S. states of Colorado and Mississippi with GPT-4.1 and Gemini-2.5-Pro. We use the generated data in an IPF-based synthesis pipeline and evaluate the resulting census tract-level synthetic populations against external benchmarks. Results show both LLMs capture several major state-level contrasts, indicating zero-shot generation produces geographically differentiated survey data. However, performance is strongly variable-dependent. Downstream effects in population synthesis are mixed, as IPF sometimes amplifies or reduces errors in the generated data. Spatial validation shows that LLM-based populations reproduce census tract-level patterns reasonably well, especially for variables that were more aligned with the ground truth data. Overall, the LLM-generated survey data shows promise as supplementary input, but not yet as a replacement for real survey data.