The Impact of Persona-based Political Perspectives on Hateful Content Detection
This work addresses the problem of reducing computational costs for achieving fairness in hate speech detection, but it is incremental as it builds on existing persona-based methods without introducing new techniques.
The study investigated whether persona-based prompting, as a less resource-intensive alternative to political pretraining, could achieve comparable results for hate speech detection in memes, finding that political biases in LLMs had little impact on classification decisions, with no significant correlation between political positioning and outcomes.
While pretraining language models with politically diverse content has been shown to improve downstream task fairness, such approaches require significant computational resources often inaccessible to many researchers and organizations. Recent work has established that persona-based prompting can introduce political diversity in model outputs without additional training. However, it remains unclear whether such prompting strategies can achieve results comparable to political pretraining for downstream tasks. We investigate this question using persona-based prompting strategies in multimodal hate-speech detection tasks, specifically focusing on hate speech in memes. Our analysis reveals that when mapping personas onto a political compass and measuring persona agreement, inherent political positioning has surprisingly little correlation with classification decisions. Notably, this lack of correlation persists even when personas are explicitly injected with stronger ideological descriptors. Our findings suggest that while LLMs can exhibit political biases in their responses to direct political questions, these biases may have less impact on practical classification tasks than previously assumed. This raises important questions about the necessity of computationally expensive political pretraining for achieving fair performance in downstream tasks.