LLMs are Better Than You Think: Label-Guided In-Context Learning for Named Entity Recognition
This addresses the challenge of low-resource NER for NLP practitioners by offering a robust, incremental improvement over existing ICL methods.
The paper tackled the problem of suboptimal demonstration retrieval in in-context learning for Named Entity Recognition by introducing DEER, a training-free method that uses label-grounded statistics to improve entity predictions, achieving performance comparable to supervised fine-tuning on five datasets across four LLMs.
In-context learning (ICL) enables large language models (LLMs) to perform new tasks using only a few demonstrations. However, in Named Entity Recognition (NER), existing ICL methods typically rely on task-agnostic semantic similarity for demonstration retrieval, which often yields less relevant examples and leads to inferior results. We introduce DEER, a training-free ICL approach that enables LLMs to make more informed entity predictions through the use of label-grounded statistics. DEER leverages token-level statistics from training labels to identify tokens most informative for entity recognition, enabling entity-focused demonstrations. It further uses these statistics to detect and refine error-prone tokens through a targeted reflection step. Evaluated on five NER datasets across four LLMs, DEER consistently outperforms existing ICL methods and achieves performance comparable to supervised fine-tuning. Further analyses demonstrate that DEER improves example retrieval, remains effective on both seen and unseen entities, and exhibits strong robustness in low-resource settings.