Scaling Native Language Identification with Transformer Adapters
This work addresses practical deployment challenges for NLI in applications like marketing and security, but it is incremental as it builds on existing transformer methods.
The paper tackled the problem of scaling native language identification (NLI) for production by addressing memory and speed limitations in transformer-based methods, introducing transformer adapters to improve efficiency while maintaining competitive performance on benchmark datasets.
Native language identification (NLI) is the task of automatically identifying the native language (L1) of an individual based on their language production in a learned language. It is useful for a variety of purposes including marketing, security and educational applications. NLI is usually framed as a multi-label classification task, where numerous designed features are combined to achieve state-of-the-art results. Recently deep generative approach based on transformer decoders (GPT-2) outperformed its counterparts and achieved the best results on the NLI benchmark datasets. We investigate this approach to determine the practical implications compared to traditional state-of-the-art NLI systems. We introduce transformer adapters to address memory limitations and improve training/inference speed to scale NLI applications for production.