A survey of diversity quantification in natural language processing: The why, what, where and how
This work addresses the ad hoc and inconsistent treatment of diversity in NLP, providing a systematized approach that could enhance understanding and comparability across methods, though it is incremental as a survey and framework proposal.
The paper surveys diversity quantification in NLP, analyzing ACL Anthology articles to identify inconsistent terminology and propose a unified taxonomy based on ecological and economic frameworks, aiming to improve formalization and comparability.
The concept of diversity has received increased consideration in Natural Language Processing (NLP) in recent years. This is due to various motivations like promoting and inclusion, approximating human linguistic behavior, and increasing systems' performance. Diversity has however often been addressed in an ad hoc manner in NLP, and with few explicit links to other domains where this notion is better theorized. We survey articles in the ACL Anthology from the past 6 years, with "diversity" or "diverse" in their title. We find a wide range of settings in which diversity is quantified, often highly specialized and using inconsistent terminology. We put forward a unified taxonomy of why, what on, where, and how diversity is measured in NLP. Diversity measures are cast upon a unified framework from ecology and economy (Stirling, 2007) with 3 dimensions of diversity: variety, balance and disparity. We discuss the trends which emerge due to this systematized approach. We believe that this study paves the way towards a better formalization of diversity in NLP, which should bring a better understanding of this notion and a better comparability between various approaches.