Less Context, Same Performance: A RAG Framework for Resource-Efficient LLM-Based Clinical NLP
This provides a resource-efficient solution for analyzing lengthy clinical documents, reducing token usage and computational costs without sacrificing accuracy, though it is incremental as it applies an existing RAG method to a specific domain.
This study tackled the challenge of long text classification for LLMs in clinical NLP by using a RAG framework to process only the most relevant text segments, finding that it matched the performance of whole-text processing with no statistically significant differences in metrics like AUC ROC and F1.
Long text classification is challenging for Large Language Models (LLMs) due to token limits and high computational costs. This study explores whether a Retrieval Augmented Generation (RAG) approach using only the most relevant text segments can match the performance of processing entire clinical notes with large context LLMs. We begin by splitting clinical documents into smaller chunks, converting them into vector embeddings, and storing these in a FAISS index. We then retrieve the top 4,000 words most pertinent to the classification query and feed these consolidated segments into an LLM. We evaluated three LLMs (GPT4o, LLaMA, and Mistral) on a surgical complication identification task. Metrics such as AUC ROC, precision, recall, and F1 showed no statistically significant differences between the RAG based approach and whole-text processing (p > 0.05p > 0.05). These findings indicate that RAG can significantly reduce token usage without sacrificing classification accuracy, providing a scalable and cost effective solution for analyzing lengthy clinical documents.