The News Comment Gap and Algorithmic Agenda Setting in Online Forums
This research addresses the problem of aligning comment ranking algorithms with journalistic and reader preferences to foster engaging online discourse, which is incremental as it builds on known gaps in news content analysis.
The study analyzed 1.2 million comments from an Austrian newspaper to investigate the 'News Comment Gap', finding that journalists prefer positive, timely, and complex comments, while readers favor comments similar to article content from elite authors, and introduced the FORUM metric to show dramatic differences in how ranking algorithms prioritize comment display.
The disparity between news stories valued by journalists and those preferred by readers, known as the "News Gap", is well-documented. However, the difference in expectations regarding news related user-generated content is less studied. Comment sections, hosted by news websites, are popular venues for reader engagement, yet still subject to editorial decisions. It is thus important to understand journalist vs reader comment preferences and how these are served by various comment ranking algorithms that represent discussions differently. We analyse 1.2 million comments from Austrian newspaper Der Standard to understand the "News Comment Gap" and the effects of different ranking algorithms. We find that journalists prefer positive, timely, complex, direct responses, while readers favour comments similar to article content from elite authors. We introduce the versatile Feature-Oriented Ranking Utility Metric (FORUM) to assess the impact of different ranking algorithms and find dramatic differences in how they prioritise the display of comments by sentiment, topical relevance, lexical diversity, and readability. Journalists can exert substantial influence over the discourse through both curatorial and algorithmic means. Understanding these choices' implications is vital in fostering engaging and civil discussions while aligning with journalistic objectives, especially given the increasing legal scrutiny and societal importance of online discourse.