Reasoning Boosts Opinion Alignment in LLMs
This work addresses the challenge of building more accurate political digital twins using LLMs, but it is incremental as it builds on existing reasoning methods without fully solving bias.
The paper tackled the problem of biased opinions in large language models (LLMs) when used for opinion modeling, and found that training models with structured reasoning improved opinion alignment, achieving competitive results on political datasets from the U.S., Europe, and Switzerland, though it did not fully eliminate bias.
Opinion modeling aims to capture individual or group political preferences, enabling applications such as digital democracies, where models could help shape fairer and more popular policies. Given their versatility, strong generalization capabilities, and demonstrated success across diverse text-to-text applications, large language models (LLMs) are natural candidates for this task. However, due to their statistical nature and limited causal understanding, they tend to produce biased opinions when prompted naively. In this work, we study whether reasoning can improve opinion alignment. Motivated by the recent advancement in mathematical reasoning enabled by reinforcement learning (RL), we train models to produce profile-consistent answers through structured reasoning. We evaluate our approach on three datasets covering U.S., European, and Swiss politics. Results indicate that reasoning enhances opinion modeling and is competitive with strong baselines, but does not fully remove bias, highlighting the need for additional mechanisms to build faithful political digital twins using LLMs. By releasing both our method and datasets, we establish a solid baseline to support future research on LLM opinion alignment.