Towards Lexical Gender Inference: A Scalable Methodology using Online Databases
This addresses the need for dynamic and high-coverage gender bias analysis in NLP, though it is incremental as it builds on existing dictionary-based approaches.
The paper tackles the problem of manually compiling lexicons for gender bias evaluation in NLP by introducing a scalable, dictionary-based method to automatically detect lexical gender in language datasets, achieving over 80% accuracy on tests with Wikipedia and existing gendered word lists.
This paper presents a new method for automatically detecting words with lexical gender in large-scale language datasets. Currently, the evaluation of gender bias in natural language processing relies on manually compiled lexicons of gendered expressions, such as pronouns ('he', 'she', etc.) and nouns with lexical gender ('mother', 'boyfriend', 'policewoman', etc.). However, manual compilation of such lists can lead to static information if they are not periodically updated and often involve value judgments by individual annotators and researchers. Moreover, terms not included in the list fall out of the range of analysis. To address these issues, we devised a scalable, dictionary-based method to automatically detect lexical gender that can provide a dynamic, up-to-date analysis with high coverage. Our approach reaches over 80% accuracy in determining the lexical gender of nouns retrieved randomly from a Wikipedia sample and when testing on a list of gendered words used in previous research.