GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer
It addresses the need for flexible and efficient NER in resource-limited scenarios, offering a practical alternative to large language models.
The paper tackles the problem of named entity recognition (NER) by introducing GLiNER, a compact model that identifies any entity type, outperforming ChatGPT and fine-tuned LLMs in zero-shot evaluations on various benchmarks.
Named Entity Recognition (NER) is essential in various Natural Language Processing (NLP) applications. Traditional NER models are effective but limited to a set of predefined entity types. In contrast, Large Language Models (LLMs) can extract arbitrary entities through natural language instructions, offering greater flexibility. However, their size and cost, particularly for those accessed via APIs like ChatGPT, make them impractical in resource-limited scenarios. In this paper, we introduce a compact NER model trained to identify any type of entity. Leveraging a bidirectional transformer encoder, our model, GLiNER, facilitates parallel entity extraction, an advantage over the slow sequential token generation of LLMs. Through comprehensive testing, GLiNER demonstrate strong performance, outperforming both ChatGPT and fine-tuned LLMs in zero-shot evaluations on various NER benchmarks.