Beyond Sentiment: Leveraging Topic Metrics for Political Stance Classification
This addresses the limitation of sentiment analysis for political science researchers by providing a more accurate method for stance classification in context-rich texts.
This study tackled the problem of political stance classification by introducing topic metrics as an alternative to sentiment metrics, showing that BERTopic improves topic coherence by 17.07% to 54.20% over traditional methods and topic metrics outperform sentiment metrics by up to 18.95% in classification performance.
Sentiment analysis, widely critiqued for capturing merely the overall tone of a corpus, falls short in accurately reflecting the latent structures and political stances within texts. This study introduces topic metrics, dummy variables converted from extracted topics, as both an alternative and complement to sentiment metrics in stance classification. By employing three datasets identified by Bestvater and Monroe (2023), this study demonstrates BERTopic's proficiency in extracting coherent topics and the effectiveness of topic metrics in stance classification. The experiment results show that BERTopic improves coherence scores by 17.07% to 54.20% when compared to traditional approaches such as Dirichlet Allocation (LDA) and Non-negative Matrix Factorization (NMF), prevalent in earlier political science research. Additionally, our results indicate topic metrics outperform sentiment metrics in stance classification, increasing performance by as much as 18.95%. Our findings suggest topic metrics are especially effective for context-rich texts and corpus where stance and sentiment correlations are weak. The combination of sentiment and topic metrics achieve an optimal performance in most of the scenarios and can further address the limitations of relying solely on sentiment as well as the low coherence score of topic metrics.