Capturing Topic Framing via Masked Language Modeling
This addresses the need for scalable framing measurement in media, which is important for understanding divergent world views, but it is incremental as it builds on existing language model techniques.
The paper tackled the problem of measuring differential framing in media by proposing a framework using masked language modeling, and showed it could capture framing with high reliability on a dataset of articles covering five polarized topics.
Differential framing of issues can lead to divergent world views on important issues. This is especially true in domains where the information presented can reach a large audience, such as traditional and social media. Scalable and reliable measurement of such differential framing is an important first step in addressing them. In this work, based on the intuition that framing affects the tone and word choices in written language, we propose a framework for modeling the differential framing of issues through masked token prediction via large-scale fine-tuned language models (LMs). Specifically, we explore three key factors for our framework: 1) prompt generation methods for the masked token prediction; 2) methods for normalizing the output of fine-tuned LMs; 3) robustness to the choice of pre-trained LMs used for fine-tuning. Through experiments on a dataset of articles from traditional media outlets covering five diverse and politically polarized topics, we show that our framework can capture differential framing of these topics with high reliability.