Examining Racial Bias in an Online Abuse Corpus with Structural Topic Modeling
This addresses bias in hate speech detection datasets, which can lead to unfair outcomes for marginalized groups, though it is incremental as it applies an existing method to a specific domain.
The researchers investigated racial bias in an online abuse corpus by applying structural topic modeling to tweets, finding that certain topics are disproportionately racialized and labeled as abusive, with concrete evidence from topic prevalence analysis.
We use structural topic modeling to examine racial bias in data collected to train models to detect hate speech and abusive language in social media posts. We augment the abusive language dataset by adding an additional feature indicating the predicted probability of the tweet being written in African-American English. We then use structural topic modeling to examine the content of the tweets and how the prevalence of different topics is related to both abusiveness annotation and dialect prediction. We find that certain topics are disproportionately racialized and considered abusive. We discuss how topic modeling may be a useful approach for identifying bias in annotated data.