Inflection-Tolerant Ontology-Based Named Entity Recognition for Real-Time Applications
This work addresses the need for fast and accurate NER in real-time systems like chatbots, but it is incremental as it builds on existing ontology-based methods to handle inflection.
The paper tackles the problem of performing Named Entity Recognition (NER) in real-time applications by addressing word inflection variations, particularly in German, using an ontology-based approach with language sources like Wiktionary. The result is a method that processes the German Wikipedia (9.4B characters) in under an hour, achieving higher precision and recall than comparably fast methods, narrowing the quality gap with slower NLP pipelines.
A growing number of applications users daily interact with have to operate in (near) real-time: chatbots, digital companions, knowledge work support systems -- just to name a few. To perform the services desired by the user, these systems have to analyze user activity logs or explicit user input extremely fast. In particular, text content (e.g. in form of text snippets) needs to be processed in an information extraction task. Regarding the aforementioned temporal requirements, this has to be accomplished in just a few milliseconds, which limits the number of methods that can be applied. Practically, only very fast methods remain, which on the other hand deliver worse results than slower but more sophisticated Natural Language Processing (NLP) pipelines. In this paper, we investigate and propose methods for real-time capable Named Entity Recognition (NER). As a first improvement step we address are word variations induced by inflection, for example present in the German language. Our approach is ontology-based and makes use of several language information sources like Wiktionary. We evaluated it using the German Wikipedia (about 9.4B characters), for which the whole NER process took considerably less than an hour. Since precision and recall are higher than with comparably fast methods, we conclude that the quality gap between high speed methods and sophisticated NLP pipelines can be narrowed a bit more without losing too much runtime performance.