Optimization of Retrieval-Augmented Generation Context with Outlier Detection
This addresses the issue of inefficient context handling in question-answering systems for users of RAG models, but it is incremental as it builds on existing outlier detection techniques.
The paper tackles the problem of irrelevant documents in retrieval-augmented generation causing hallucinations and reduced LLM performance by proposing outlier detection methods to select semantically relevant documents, finding that improvements were greatest with increasing question and answer complexity.
In this paper, we focus on methods to reduce the size and improve the quality of the prompt context required for question-answering systems. Attempts to increase the number of retrieved chunked documents and thereby enlarge the context related to the query can significantly complicate the processing and decrease the performance of a Large Language Model (LLM) when generating responses to queries. It is well known that a large set of documents retrieved from a database in response to a query may contain irrelevant information, which often leads to hallucinations in the resulting answers. Our goal is to select the most semantically relevant documents, treating the discarded ones as outliers. We propose and evaluate several methods for identifying outliers by creating features that utilize the distances of embedding vectors, retrieved from the vector database, to both the centroid and the query vectors. The methods were evaluated by comparing the similarities of the retrieved LLM responses to ground-truth answers obtained using the OpenAI GPT-4o model. It was found that the greatest improvements were achieved with increasing complexity of the questions and answers.