VicunaNER: Zero/Few-shot Named Entity Recognition using Vicuna
This work addresses data privacy and reproducibility issues for researchers and practitioners using NER, though it is incremental as it adapts an existing open-source model to a specific task.
The authors tackled the problem of data leakage and non-reproducibility in zero/few-shot Named Entity Recognition (NER) by proposing VicunaNER, a framework based on the open-source LLM Vicuna, which achieved superior performance on 10 datasets across 5 domains in zero-shot and on Few-NERD in few-shot settings.
Large Language Models (LLMs, e.g., ChatGPT) have shown impressive zero- and few-shot capabilities in Named Entity Recognition (NER). However, these models can only be accessed via online APIs, which may cause data leak and non-reproducible problems. In this paper, we propose VicunaNER, a zero/few-shot NER framework based on the newly released open-source LLM -- Vicuna. VicunaNER is a two-phase framework, where each phase leverages multi-turn dialogues with Vicuna to recognize entities from texts. We name the second phase as Re-Recognition, which recognizes those entities not recognized in the first phase (a.k.a. Recognition). Moreover, we set entity correctness check dialogues in each phase to filter out wrong entities. We evaluate VicunaNER's zero-shot capacity on 10 datasets crossing 5 domains and few-shot capacity on Few-NERD. Experimental results demonstrate that VicunaNER achieves superior performance in both shot settings. Additionally, we conduct comprehensive investigations on Vicuna from multiple perspectives.