Hybrid moderation in the newsroom: Recommending featured posts to content moderators
This work addresses the time-consuming task of content moderation for news outlets, but it is incremental as it applies existing methods to a specific domain.
The paper tackles the problem of moderating user-generated comments in online news outlets by developing a recommender system to assist content moderators in selecting featured posts, achieving an F1-score of 0.44 and NDCG scores up to 0.87.
Online news outlets are grappling with the moderation of user-generated content within their comment section. We present a recommender system based on ranking class probabilities to support and empower the moderator in choosing featured posts, a time-consuming task. By combining user and textual content features we obtain an optimal classification F1-score of 0.44 on the test set. Furthermore, we observe an optimum mean NDCG@5 of 0.87 on a large set of validation articles. As an expert evaluation, content moderators assessed the output of a random selection of articles by choosing comments to feature based on the recommendations, which resulted in a NDCG score of 0.83. We conclude that first, adding text features yields the best score and second, while choosing featured content remains somewhat subjective, content moderators found suitable comments in all but one evaluated recommendations. We end the paper by analyzing our best-performing model, a step towards transparency and explainability in hybrid content moderation.