LLM-based Prompt Ensemble for Reliable Medical Entity Recognition from EHRs
It addresses the problem of extracting medical entities from unstructured clinical text for healthcare applications, but it is incremental as it applies existing LLM and ensemble techniques to this domain.
This paper tackled medical entity recognition from electronic health records using large language models with prompt engineering, achieving an F1-score of 0.95 and recall of 0.98 with GPT-4o and an ensemble method.
Electronic Health Records (EHRs) are digital records of patient information, often containing unstructured clinical text. Named Entity Recognition (NER) is essential in EHRs for extracting key medical entities like problems, tests, and treatments to support downstream clinical applications. This paper explores prompt-based medical entity recognition using large language models (LLMs), specifically GPT-4o and DeepSeek-R1, guided by various prompt engineering techniques, including zero-shot, few-shot, and an ensemble approach. Among all strategies, GPT-4o with prompt ensemble achieved the highest classification performance with an F1-score of 0.95 and recall of 0.98, outperforming DeepSeek-R1 on the task. The ensemble method improved reliability by aggregating outputs through embedding-based similarity and majority voting.