JRC-Names: A freely available, highly multilingual named entity resource
This provides a large-scale, multilingual dataset for NLP researchers and practitioners, though it is incremental as it builds on existing data sources like news and Wikipedia.
The authors compiled a freely available, highly multilingual named entity resource with 205,000 person and organization names and variants across over 20 scripts, derived from seven years of multilingual news analysis and Wikipedia mining, to improve tasks like name search, named entity recognition, and machine translation.
This paper describes a new, freely available, highly multilingual named entity resource for person and organisation names that has been compiled over seven years of large-scale multilingual news analysis combined with Wikipedia mining, resulting in 205,000 per-son and organisation names plus about the same number of spelling variants written in over 20 different scripts and in many more languages. This resource, produced as part of the Europe Media Monitor activity (EMM, http://emm.newsbrief.eu/overview.html), can be used for a number of purposes. These include improving name search in databases or on the internet, seeding machine learning systems to learn named entity recognition rules, improve machine translation results, and more. We describe here how this resource was created; we give statistics on its current size; we address the issue of morphological inflection; and we give details regarding its functionality. Updates to this resource will be made available daily.