POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection
This provides a tool for political science researchers and analysts to characterize ideology in diverse texts, though it is incremental as it builds on existing pretrained language models.
The paper tackles the lack of general-purpose tools for predicting ideology across text genres by pretraining a language model with ideology-driven objectives using same-story article comparisons, achieving state-of-the-art results on ideology prediction and stance detection tasks.
Ideology is at the core of political science research. Yet, there still does not exist general-purpose tools to characterize and predict ideology across different genres of text. To this end, we study Pretrained Language Models using novel ideology-driven pretraining objectives that rely on the comparison of articles on the same story written by media of different ideologies. We further collect a large-scale dataset, consisting of more than 3.6M political news articles, for pretraining. Our model POLITICS outperforms strong baselines and the previous state-of-the-art models on ideology prediction and stance detection tasks. Further analyses show that POLITICS is especially good at understanding long or formally written texts, and is also robust in few-shot learning scenarios.