Who is Addressed in this Comment? Automatically Classifying Meta-Comments in News Comments
This addresses the challenge for newsrooms overwhelmed by comment volume, enabling more efficient moderation, though it is incremental as it applies existing classification methods to a specific domain.
The paper tackled the problem of automatically identifying meta-comments in news comments that address newsrooms or journalists, achieving F0.5 scores between 76% and 91% on datasets from German and Austrian newspapers.
User comments have become an essential part of online journalism. However, newsrooms are often overwhelmed by the vast number of diverse comments, for which a manual analysis is barely feasible. Identifying meta-comments that address or mention newsrooms, individual journalists, or moderators and that may call for reactions is particularly critical. In this paper, we present an automated approach to identify and classify meta-comments. We compare comment classification based on manually extracted features with an end-to-end learning approach. We develop, optimize, and evaluate multiple classifiers on a comment dataset of the large German online newsroom SPIEGEL Online and the 'One Million Posts' corpus of DER STANDARD, an Austrian newspaper. Both optimized classification approaches achieved encouraging $F_{0.5}$ values between 76% and 91%. We report on the most significant classification features with the results of a qualitative analysis and discuss how our work contributes to making participation in online journalism more constructive.