Second Order WinoBias (SoWinoBias) Test Set for Latent Gender Bias Detection in Coreference Resolution
This addresses bias detection in NLP for researchers and practitioners, but it is incremental as it builds on existing test sets like WinoBias.
The authors tackled latent gender bias in coreference resolution systems by creating the SoWinoBias test set to measure bias without explicit gender words, and they evaluated current debiasing methods on it, finding that these methods often fail to address such latent biases effectively.
We observe an instance of gender-induced bias in a downstream application, despite the absence of explicit gender words in the test cases. We provide a test set, SoWinoBias, for the purpose of measuring such latent gender bias in coreference resolution systems. We evaluate the performance of current debiasing methods on the SoWinoBias test set, especially in reference to the method's design and altered embedding space properties. See https://github.com/hillarydawkins/SoWinoBias.