Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encoding
This addresses the issue of NLP performance degradation for speakers of World Englishes like Colloquial Singapore English and African American Vernacular English, representing an incremental improvement with specific gains.
The paper tackled the problem of NLP systems' lack of robustness to inflectional variation in non-standard Englishes by proposing Base-Inflection Encoding (BITE), which improved generalization to dialects without explicit training, enhanced vocabulary efficiency, and accelerated translation model convergence.
Inflectional variation is a common feature of World Englishes such as Colloquial Singapore English and African American Vernacular English. Although comprehension by human readers is usually unimpaired by non-standard inflections, current NLP systems are not yet robust. We propose Base-Inflection Encoding (BITE), a method to tokenize English text by reducing inflected words to their base forms before reinjecting the grammatical information as special symbols. Fine-tuning pretrained NLP models for downstream tasks using our encoding defends against inflectional adversaries while maintaining performance on clean data. Models using BITE generalize better to dialects with non-standard inflections without explicit training and translation models converge faster when trained with BITE. Finally, we show that our encoding improves the vocabulary efficiency of popular data-driven subword tokenizers. Since there has been no prior work on quantitatively evaluating vocabulary efficiency, we propose metrics to do so.