Mohammad Hammas Saeed

Mohammad Hammas Saeed, David A. Broniatowski, Joseph Simons et al.

Social media platforms shape public discourse through two fundamental design choices that naturally co-occur in any field investigation: platform architecture, which defines what types of actors exist and how they interact, and recommendation algorithm, which determines what content is surfaced to users. Using agent-based simulation, we orthogonally manipulate both factors, exploring four prototypical architectures -- tree (e.g., Reddit), layered hierarchy (e.g., Facebook), network (e.g., Twitter), and complete graph (e.g., TikTok) -- and two algorithms: chronological (LIFO) and popularity-based (Hot). Drawing on prior theory that identifies and ranks canonical system architectures in terms of their flexibility we hypothesize that algorithmic effects on information spread and quality should be largest on the most flexible platforms and smallest on the most constrained ones. We find strong confirmation of this prediction. On tree-like platforms like Reddit, the algorithm has no detectable effect on information spread and quality. On layered hierarchies and networks like Facebook and Twitter, respectively, the Hot algorithm has modest positive effects on both the spread of information and its quality. On complete structures like TikTok, the Hot algorithm leads to a winner-take-all dynamics that has strong negative effects on both information spread and quality, making the relation between content quality and popularity unpredictable. These findings imply that architectural considerations are more powerful levers than algorithmic interventions for the design of healthy online spaces and public discourse. Platform reform efforts focused exclusively on algorithm choice may be insufficient on architecturally unconstrained platforms and unnecessary on architecturally constrained ones.

Pujan Paudel, Mohammad Hammas Saeed, Rebecca Auger et al.

Automated soft moderation systems are unable to ascertain if a post supports or refutes a false claim, resulting in a large number of contextual false positives. This limits their effectiveness, for example undermining trust in health experts by adding warnings to their posts or resorting to vague warnings instead of granular fact-checks, which result in desensitizing users. In this paper, we propose to incorporate stance detection into existing automated soft-moderation pipelines, with the goal of ruling out contextual false positives and providing more precise recommendations for social media content that should receive warnings. We develop a textual deviation task called Contrastive Textual Deviation (CTD) and show that it outperforms existing stance detection approaches when applied to soft moderation.We then integrate CTD into the stateof-the-art system for automated soft moderation Lambretta, showing that our approach can reduce contextual false positives from 20% to 2.1%, providing another important building block towards deploying reliable automated soft moderation tools on social media.

Mohammad Hammas Saeed, Shiza Ali, Jeremy Blackburn et al.

Growing evidence points to recurring influence campaigns on social media, often sponsored by state actors aiming to manipulate public opinion on sensitive political topics. Typically, campaigns are performed through instrumented accounts, known as troll accounts; despite their prominence, however, little work has been done to detect these accounts in the wild. In this paper, we present TROLLMAGNIFIER, a detection system for troll accounts. Our key observation, based on analysis of known Russian-sponsored troll accounts identified by Reddit, is that they show loose coordination, often interacting with each other to further specific narratives. Therefore, troll accounts controlled by the same actor often show similarities that can be leveraged for detection. TROLLMAGNIFIER learns the typical behavior of known troll accounts and identifies more that behave similarly. We train TROLLMAGNIFIER on a set of 335 known troll accounts and run it on a large dataset of Reddit accounts. Our system identifies 1,248 potential troll accounts; we then provide a multi-faceted analysis to corroborate the correctness of our classification. In particular, 66% of the detected accounts show signs of being instrumented by malicious actors (e.g., they were created on the same exact day as a known troll, they have since been suspended by Reddit, etc.). They also discuss similar topics as the known troll accounts and exhibit temporal synchronization in their activity. Overall, we show that using TROLLMAGNIFIER, one can grow the initial knowledge of potential trolls provided by Reddit by over 300%.