Gaussian mixture models as a proxy for interacting language models
This provides a more efficient proxy for social science simulations where large-scale experiments are infeasible, though it is incremental as it builds on existing LLM frameworks.
The paper tackles the problem of using large language models (LLMs) for studying human behavior in social sciences, which is computationally expensive, by proposing interacting Gaussian mixture models (GMMs) as a simpler alternative that captures key dynamics of interacting LLMs.
Large language models (LLMs) are a powerful tool with the ability to match human capabilities and behavior in many settings. Retrieval-augmented generation (RAG) further allows LLMs to generate diverse output depending on the contents of their RAG database. This motivates their use in the social sciences to study human behavior between individuals when large-scale experiments are infeasible. However, LLMs depend on complex, computationally expensive algorithms. In this paper, we introduce interacting Gaussian mixture models (GMMs) as an alternative to similar frameworks using LLMs. We compare a simplified model of GMMs to select experimental simulations of LLMs whose updating and response depend on feedback from other LLMs. We find that interacting GMMs capture important features of the dynamics in interacting LLMs, and we investigate key similarities and differences between interacting LLMs and GMMs. We conclude by discussing the benefits of Gaussian mixture models, potential modifications, and future research directions.