Classifying Problem and Solution Framing in Congressional Social Media
This work addresses the need for automated analysis of policy framing in congressional social media, which is incremental as it applies existing NLP methods to a new domain-specific dataset.
The researchers tackled the problem of automatically classifying US Senator tweets as either problem-focused or solution-focused, achieving an average weighted F1 score above 0.8 using a BERTweet Base model on a dataset of 1.68 million tweets.
Policy setting in the USA according to the ``Garbage Can'' model differentiates between ``problem'' and ``solution'' focused processes. In this paper, we study a large dataset of US Senator postings on Twitter (1.68m tweets in total). Our objective is to develop an automated method to label Senatorial posts as either in the problem or solution streams. Two academic policy experts labeled a subset of 3967 tweets as either problem, solution, or other (anything not problem or solution). We split off a subset of 500 tweets into a test set, with the remaining 3467 used for training. During development, this training set was further split by 60/20/20 proportions for fitting, validation, and development test sets. We investigated supervised learning methods for building problem/solution classifiers directly on the training set, evaluating their performance in terms of F1 score on the validation set, allowing us to rapidly iterate through models and hyperparameters, achieving an average weighted F1 score of above 0.8 on cross validation across the three categories using a BERTweet Base model.