Counterfactual Probing for the Influence of Affect and Specificity on Intergroup Bias
This work addresses bias in NLP by exploring pragmatic features in intergroup contexts, but it is incremental as it builds on existing framing with preliminary findings.
The paper investigates whether specificity and affect in tweets systematically vary across intergroup contexts, linking a revised framing of bias to language output, and finds modest correlations and that neural models use affect reliably but specificity inconclusively for classification.
While existing work on studying bias in NLP focues on negative or pejorative language use, Govindarajan et al. (2023) offer a revised framing of bias in terms of intergroup social context, and its effects on language behavior. In this paper, we investigate if two pragmatic features (specificity and affect) systematically vary in different intergroup contexts -- thus connecting this new framing of bias to language output. Preliminary analysis finds modest correlations between specificity and affect of tweets with supervised intergroup relationship (IGR) labels. Counterfactual probing further reveals that while neural models finetuned for predicting IGR labels reliably use affect in classification, the model's usage of specificity is inconclusive. Code and data can be found at: https://github.com/venkatasg/intergroup-probing