LLMs Can Infer Political Alignment from Online Conversations
This work addresses a fundamental privacy risk for individuals online by demonstrating the capacity of LLMs to exploit socio-cultural correlates, highlighting potential misuse risks.
The study tackled the problem of inferring hidden political alignment from online conversations using large language models (LLMs), showing that LLMs significantly outperform traditional machine learning models in prediction accuracy, with improvements from aggregating text-level inferences and using politics-adjacent domains.
Due to the correlational structure in our traits such as identities, cultures, and political attitudes, seemingly innocuous preferences such as following a band or using a specific slang, can reveal private traits. This possibility, especially when combined with massive, public social data and advanced computational methods, poses a fundamental privacy risk. Given our increasing data exposure online and the rapid advancement of AI are increasing the misuse potential of such risk, it is therefore critical to understand capacity of large language models (LLMs) to exploit it. Here, using online discussions on Debate.org and Reddit, we show that LLMs can reliably infer hidden political alignment, significantly outperforming traditional machine learning models. Prediction accuracy further improves as we aggregate multiple text-level inferences into a user-level prediction, and as we use more politics-adjacent domains. We demonstrate that LLMs leverage the words that can be highly predictive of political alignment while not being explicitly political. Our findings underscore the capacity and risks of LLMs for exploiting socio-cultural correlates.