Can Large Language Models Predict Associations Among Human Attitudes?
This research addresses the challenge of understanding human belief systems for psychologists and AI researchers, but it is incremental as it builds on prior work by extending predictions to dissimilar attitudes.
The study tackled the problem of predicting human attitudes using large language models, specifically testing GPT-4o's ability to predict associations among attitudes even without surface similarity, and found that it could recreate pairwise correlations and predict attitudes effectively, with surface similarity improving accuracy but not being essential.
Prior work has shown that large language models (LLMs) can predict human attitudes based on other attitudes, but this work has largely focused on predictions from highly similar and interrelated attitudes. In contrast, human attitudes are often strongly associated even across disparate and dissimilar topics. Using a novel dataset of human responses toward diverse attitude statements, we found that a frontier language model (GPT-4o) was able to recreate the pairwise correlations among individual attitudes and to predict individuals' attitudes from one another. Crucially, in an advance over prior work, we tested GPT-4o's ability to predict in the absence of surface-similarity between attitudes, finding that while surface similarity improves prediction accuracy, the model was still highly-capable of generating meaningful social inferences between dissimilar attitudes. Altogether, our findings indicate that LLMs capture crucial aspects of the deeper, latent structure of human belief systems.