Retrieval augmented generation based dynamic prompting for few-shot biomedical named entity recognition using large language models
This work addresses biomedical NLP tasks for researchers and practitioners, offering incremental improvements in few-shot learning through enhanced prompting techniques.
The paper tackled performance challenges of large language models for few-shot biomedical named entity recognition by investigating a dynamic prompting strategy using retrieval-augmented generation, resulting in improved average F1-scores by up to 12% with static prompting and further gains of 7.3% and 5.6% with dynamic prompting in 5-shot and 10-shot settings.
Biomedical named entity recognition (NER) is a high-utility natural language processing (NLP) task, and large language models (LLMs) show promise particularly in few-shot settings (i.e., limited training data). In this article, we address the performance challenges of LLMs for few-shot biomedical NER by investigating a dynamic prompting strategy involving retrieval-augmented generation (RAG). In our approach, the annotated in-context learning examples are selected based on their similarities with the input texts, and the prompt is dynamically updated for each instance during inference. We implemented and optimized static and dynamic prompt engineering techniques and evaluated them on five biomedical NER datasets. Static prompting with structured components increased average F1-scores by 12% for GPT-4, and 11% for GPT-3.5 and LLaMA 3-70B, relative to basic static prompting. Dynamic prompting further improved performance, with TF-IDF and SBERT retrieval methods yielding the best results, improving average F1-scores by 7.3% and 5.6% in 5-shot and 10-shot settings, respectively. These findings highlight the utility of contextually adaptive prompts via RAG for biomedical NER.