Topic Bias in Emotion Classification
This work addresses a data quality issue for researchers and practitioners in NLP and affective computing, though it is incremental as it applies an established debiasing method to a specific domain.
The paper tackles the problem of topic bias in emotion classification, showing that existing corpora have unrealistic correlations between overrepresented topics and emotion labels, which harms model generalizability, and demonstrates that adversarial correction via gradient reversal mitigates the issue.
Emotion corpora are typically sampled based on keyword/hashtag search or by asking study participants to generate textual instances. In any case, these corpora are not uniform samples representing the entirety of a domain. We hypothesize that this practice of data acquisition leads to unrealistic correlations between overrepresented topics in these corpora that harm the generalizability of models. Such topic bias could lead to wrong predictions for instances like "I organized the service for my aunt's funeral." when funeral events are over-represented for instances labeled with sadness, despite the emotion of pride being more appropriate here. In this paper, we study this topic bias both from the data and the modeling perspective. We first label a set of emotion corpora automatically via topic modeling and show that emotions in fact correlate with specific topics. Further, we see that emotion classifiers are confounded by such topics. Finally, we show that the established debiasing method of adversarial correction via gradient reversal mitigates the issue. Our work points out issues with existing emotion corpora and that more representative resources are required for fair evaluation of models predicting affective concepts from text.