Say Less, Mean More: Leveraging Pragmatics in Retrieval-Augmented Generation
This work addresses the challenge of making retrieved contexts more relevant and concise for question answering tasks, though it is incremental as it builds on existing RAG frameworks.
The paper tackles the problem of improving retrieval-augmented generation (RAG) by injecting pragmatic principles to enhance the utility of retrieved contexts, resulting in relative accuracy improvements of up to 19.7% on PubHealth and 10% on ARC-Challenge compared to conventional RAG systems.
We propose a simple, unsupervised method that injects pragmatic principles in retrieval-augmented generation (RAG) frameworks such as Dense Passage Retrieval to enhance the utility of retrieved contexts. Our approach first identifies which sentences in a pool of documents retrieved by RAG are most relevant to the question at hand, cover all the topics addressed in the input question and no more, and then highlights these sentences within their context, before they are provided to the LLM, without truncating or altering the context in any other way. We show that this simple idea brings consistent improvements in experiments on three question answering tasks (ARC-Challenge, PubHealth and PopQA) using five different LLMs. It notably enhances relative accuracy by up to 19.7% on PubHealth and 10% on ARC-Challenge compared to a conventional RAG system.