Towards Robust Named Entity Recognition for Historic German
This work addresses a domain-specific problem for researchers in historical linguistics, but it is incremental as it builds on existing pre-trained models.
The paper tackles low-resource named entity recognition for Historic German by applying pre-trained character-based language models, resulting in an improvement of up to 6% in F1 score over classical methods.
Recent advances in language modeling using deep neural networks have shown that these models learn representations, that vary with the network depth from morphology to semantic relationships like co-reference. We apply pre-trained language models to low-resource named entity recognition for Historic German. We show on a series of experiments that character-based pre-trained language models do not run into trouble when faced with low-resource datasets. Our pre-trained character-based language models improve upon classical CRF-based methods and previous work on Bi-LSTMs by boosting F1 score performance by up to 6%. Our pre-trained language and NER models are publicly available under https://github.com/stefan-it/historic-ner .