Language Models Trained on Media Diets Can Predict Public Opinion
This provides a novel tool for supplementing traditional polls and forecasting public opinion, with potential applications in social science and media studies, though it is incremental in combining existing techniques.
The authors tackled the problem of measuring public opinion by introducing a method that uses language models trained on media diets to predict survey responses, achieving results that are predictive of human judgments and more accurate for individuals who closely follow media.
Public opinion reflects and shapes societal behavior, but the traditional survey-based tools to measure it are limited. We introduce a novel approach to probe media diet models -- language models adapted to online news, TV broadcast, or radio show content -- that can emulate the opinions of subpopulations that have consumed a set of media. To validate this method, we use as ground truth the opinions expressed in U.S. nationally representative surveys on COVID-19 and consumer confidence. Our studies indicate that this approach is (1) predictive of human judgements found in survey response distributions and robust to phrasing and channels of media exposure, (2) more accurate at modeling people who follow media more closely, and (3) aligned with literature on which types of opinions are affected by media consumption. Probing language models provides a powerful new method for investigating media effects, has practical applications in supplementing polls and forecasting public opinion, and suggests a need for further study of the surprising fidelity with which neural language models can predict human responses.