Building Better: Avoiding Pitfalls in Developing Language Resources when Data is Scarce
This work addresses the problem of data scarcity and ethical concerns in NLP for under-resourced languages, offering guidance for researchers and practitioners, though it is incremental as it builds on existing critiques without introducing new technical methods.
The paper tackles the challenges of developing language resources for under-resourced languages by collecting feedback from stakeholders, identifying issues in data quality and ethical annotation practices, and providing recommendations for creating culturally appropriate and respectful language artefacts.
Language is a form of symbolic capital that affects people's lives in many ways (Bourdieu1977,1991). As a powerful means of communication, it reflects identities, cultures, traditions, and societies more broadly. Therefore, data in a given language should be regarded as more than just a collection of tokens. Rigorous data collection and labeling practices are essential for developing more human-centered and socially aware technologies. Although there has been growing interest in under-resourced languages within the NLP community, work in this area faces unique challenges, such as data scarcity and limited access to qualified annotators. In this paper, we collect feedback from individuals directly involved in and impacted by NLP artefacts for medium- and low-resource languages. We conduct both quantitative and qualitative analyses of their responses and highlight key issues related to: (1) data quality, including linguistic and cultural appropriateness; and (2) the ethics of common annotation practices, such as the misuse of participatory research. Based on these findings, we make several recommendations for creating high-quality language artefacts that reflect the cultural milieu of their speakers, while also respecting the dignity and labor of data workers.