Named Entity Recognition with Extremely Limited Data
This addresses the challenge of NER for long-tail entity classes where labeled data is scarce or costly, though it appears incremental as it builds on existing CRF methods.
The paper tackles the problem of named entity recognition (NER) for classes with extremely limited labeled data by exploring NER as a search task, where entity classes are queries and entities are documents, and investigates CRF-based models with handcrafted features for this transformation.
Traditional information retrieval treats named entity recognition as a pre-indexing corpus annotation task, allowing entity tags to be indexed and used during search. Named entity taggers themselves are typically trained on thousands or tens of thousands of examples labeled by humans. However, there is a long tail of named entities classes, and for these cases, labeled data may be impossible to find or justify financially. We propose exploring named entity recognition as a search task, where the named entity class of interest is a query, and entities of that class are the relevant "documents". What should that query look like? Can we even perform NER-style labeling with tens of labels? This study presents an exploration of CRF-based NER models with handcrafted features and of how we might transform them into search queries.