Large-Scale Contextualised Language Modelling for Norwegian
This work addresses the lack of large-scale language models for Norwegian, providing resources and benchmarks for NLP applications in this language, though it is incremental as it applies existing methods to new data.
The paper introduces the NorLM initiative, which developed the first large-scale monolingual language models for Norwegian using ELMo and BERT frameworks, and presents benchmark results on NLP tasks for Norwegian.
We present the ongoing NorLM initiative to support the creation and use of very large contextualised language models for Norwegian (and in principle other Nordic languages), including a ready-to-use software environment, as well as an experience report for data preparation and training. This paper introduces the first large-scale monolingual language models for Norwegian, based on both the ELMo and BERT frameworks. In addition to detailing the training process, we present contrastive benchmark results on a suite of NLP tasks for Norwegian. For additional background and access to the data, models, and software, please see http://norlm.nlpl.eu