Experience Retrieval-Augmentation with Electronic Health Records Enables Accurate Discharge QA
This work addresses the need for more accurate discharge-related question answering in healthcare by leveraging case-based knowledge from EHRs, representing an incremental advancement in retrieval-augmented generation for medical reasoning.
The paper tackles the problem of improving Large Language Models' reliability in clinical applications by proposing ExpRAG, a retrieval-augmentation framework based on Electronic Health Records, which outperforms a text-based ranker with an average relative improvement of 5.2% on the DischargeQA dataset.
To improve the reliability of Large Language Models (LLMs) in clinical applications, retrieval-augmented generation (RAG) is extensively applied to provide factual medical knowledge. However, beyond general medical knowledge from open-ended datasets, clinical case-based knowledge is also critical for effective medical reasoning, as it provides context grounded in real-world patient experiences.Motivated by this, we propose Experience Retrieval-Augmentation ExpRAG framework based on Electronic Health Record(EHR), aiming to offer the relevant context from other patients' discharge reports. ExpRAG performs retrieval through a coarse-to-fine process, utilizing an EHR-based report ranker to efficiently identify similar patients, followed by an experience retriever to extract task-relevant content for enhanced medical reasoning.To evaluate ExpRAG, we introduce DischargeQA, a clinical QA dataset with 1,280 discharge-related questions across diagnosis, medication, and instruction tasks. Each problem is generated using EHR data to ensure realistic and challenging scenarios. Experimental results demonstrate that ExpRAG consistently outperforms a text-based ranker, achieving an average relative improvement of 5.2%, highlighting the importance of case-based knowledge for medical reasoning.