Derek Ruths

Jana Kurrek, Haji Mohammad Saleem, Derek Ruths

Uses of pejorative expressions can be benign or actively empowering. When models for abuse detection misclassify these expressions as derogatory, they inadvertently censor productive conversations held by marginalized groups. One way to engage with non-dominant perspectives is to add context around conversations. Previous research has leveraged user- and thread-level features, but it often neglects the spaces within which productive conversations take place. Our paper highlights how community context can improve classification outcomes in abusive language detection. We make two main contributions to this end. First, we demonstrate that online communities cluster by the nature of their support towards victims of abuse. Second, we establish how community context improves accuracy and reduces the false positive rates of state-of-the-art abusive language classifiers. These findings suggest a promising direction for context-aware models in abusive language research.

Benjamin Steel, Derek Ruths

From changing fashion trends to views on world leaders and economic policies, large-scale shifts in group positions happen regularly and unexpectedly. How can we track these in the wild? How can we characterize them? Existing work has primarily leveraged stance detection to track shifts of specific groups on a single issue. However, such methods will only find shifts when they accurately pick exactly the right group and right issue. They do not capture the multi-dimensional, multi-resolution stance landscape in which these shifts actually happen. To better model drift and shift in public opinion, we require a framework that can track change at the population level, across a diverse range of issues. We propose a method to infer the potential landscape of stance dynamics, the gradient of which shows large-scale stance shifts, and apply it to show en-mass stance shifts by prominent Canadian political figures across multiple platforms and years. We do this using large-scale stance detection to find stance expressions, dimensionality reduction to find the low-dimensional linear latent space, and potential landscape neural networks to find the potential landscape of that space. This allows us to find a coherent, linear, three-dimensional space that explains 45\% of the variance in stance, where we can explain the specific characteristics of each dimension. We show that while the predictive performance is sufficient to validate its descriptive-ness, in practice its predictive performance is mixed.

Benjamin Steel, Miriam Schirmer, Derek Ruths et al.

TikTok is now a massive platform, and has a deep impact on global events. Despite preliminary studies, issues remain in determining fundamental characteristics of the platform. We develop a method to extract a representative sample of >99% of posts from a given time range on TikTok, and use it to collect all posts from a full hour on the platform, alongside all posts from a single minute from each hour of a day. Through this, we obtain post metadata, video media, and comments from a close-to-complete slice of TikTok, and report the critical statistics of the platform. Notably, we estimate a total of 269 million posts produced on the day we looked at, that 18% of videos on the platform feature children, and that at least 0.5% of posts contain artificial intelligence-generated content.

Caitrin Armstrong, Derek Ruths

In this paper we introduce Reddit legends, a collection of venerated old posts that have become famous on Reddit. To establish the utility of Reddit legends for both computational science/HCI and folkloristics, we investigate two main questions: (1) whether they can be considered folklore, i.e. if they have consistent form, cultural significance, and undergo spontaneous transmission, and (2) whether they can be studied in a systematic manner. Through several subtasks, including the creation of a typology, an analysis of references to Reddit legends, and an examination of some of the textual characteristics of referencing behaviour, we show that Reddit legends can indeed be considered as folklore and that they are amendable to systematic text-based approaches. We discuss how these results will enable future analyses of folklore on Reddit, including tracking subreddit-wide and individual-user behaviour, and the relationship of this behaviour to other cultural markers.

Logan Martel, Edward Newell, Drew Margolin et al.

On the topic of journalistic integrity, the current state of accurate, impartial news reporting has garnered much debate in context to the 2016 US Presidential Election. In pursuit of computational evaluation of news text, the statements (attributions) ascribed by media outlets to sources provide a common category of evidence on which to operate. In this paper, we develop an approach to compare partisan traits of news text attributions and apply it to characterize differences in statements ascribed to candidate, Hilary Clinton, and incumbent President, Donald Trump. In doing so, we present a model trained on over 600 in-house annotated attributions to identify each candidate with accuracy > 88%. Finally, we discuss insights from its performance for future research.

Haji Mohammad Saleem, Kelly P Dillon, Susan Benesch et al.

Online social platforms are beset with hateful speech - content that expresses hatred for a person or group of people. Such content can frighten, intimidate, or silence platform users, and some of it can inspire other users to commit violence. Despite widespread recognition of the problems posed by such content, reliable solutions even for detecting hateful speech are lacking. In the present work, we establish why keyword-based methods are insufficient for detection. We then propose an approach to detecting hateful speech that uses content produced by self-identifying hateful communities as training data. Our approach bypasses the expensive annotation process often required to train keyword systems and performs well across several established platforms, making substantial improvements over current state-of-the-art approaches.