Panning for gold: Lessons learned from the platform-agnostic automated detection of political content in textual data
This work addresses the challenge of analyzing large volumes of diverse online data for political communication research, though it is incremental as it systematically compares existing methods rather than introducing new ones.
The paper tackled the problem of automatically detecting political content in textual data across online platforms by comparing dictionary-based, supervised machine learning, and neural network techniques, finding that neural network and machine learning models performed best on less noisy data while dictionary-based models were more robust on noisy data.
The growing availability of data about online information behaviour enables new possibilities for political communication research. However, the volume and variety of these data makes them difficult to analyse and prompts the need for developing automated content approaches relying on a broad range of natural language processing techniques (e.g. machine learning- or neural network-based ones). In this paper, we discuss how these techniques can be used to detect political content across different platforms. Using three validation datasets, which include a variety of political and non-political textual documents from online platforms, we systematically compare the performance of three groups of detection techniques relying on dictionaries, supervised machine learning, or neural networks. We also examine the impact of different modes of data preprocessing (e.g. stemming and stopword removal) on the low-cost implementations of these techniques using a large set (n = 66) of detection models. Our results show the limited impact of preprocessing on model performance, with the best results for less noisy data being achieved by neural network- and machine-learning-based models, in contrast to the more robust performance of dictionary-based models on noisy data.