What Matters When Building Universal Multilingual Named Entity Recognition Models?
This work addresses the problem of unclear design decisions in multilingual NER for researchers and practitioners, though it is incremental as it builds on existing methods with systematic evaluation.
The paper tackles the lack of systematic justification in building universal multilingual NER models by conducting extensive experiments on architectures, backbones, objectives, and data composition, resulting in Otter, a model that outperforms GLiNER-x-base by 5.3pp in F1 and is competitive with large generative models while being more efficient.
Recent progress in universal multilingual named entity recognition (NER) has been driven by advances in multilingual transformer models and task-specific architectures, loss functions, and training datasets. Despite substantial prior work, we find that many critical design decisions for such models are made without systematic justification, with architectural components, training objectives, and data sources evaluated only in combination rather than in isolation. We argue that these decisions impede progress in the field by making it difficult to identify which choices improve model performance. In this work, we conduct extensive experiments around architectures, transformer backbones, training objectives, and data composition across a wide range of languages. Based on these insights, we introduce Otter, a universal multilingual NER model supporting over 100 languages. Otter achieves consistent improvements over strong multilingual NER baselines, outperforming GLiNER-x-base by 5.3pp in F1 and achieves competitive performance compared to large generative models such as Qwen3-32B, while being substantially more efficient. We release model checkpoints, training and evaluation code to facilitate reproducibility and future research.