Lightweight Relevance Grader in RAG
This addresses the challenge of computational efficiency in RAG systems for users needing accurate document retrieval, though it is incremental as it builds on existing relevance grading methods.
The paper tackled the problem of ensuring document relevance in Retrieval-Augmented Generation (RAG) by finetuning a lightweight small language model as a relevance grader, achieving a precision increase from 0.1301 to 0.7750, comparable to a much larger model.
Retrieval-Augmented Generation (RAG) addresses limitations of large language models (LLMs) by leveraging a vector database to provide more accurate and up-to-date information. When a user submits a query, RAG executes a vector search to find relevant documents, which are then used to generate a response. However, ensuring the relevance of retrieved documents with a query would be a big challenge. To address this, a secondary model, known as a relevant grader, can be served to verify its relevance. To reduce computational requirements of a relevant grader, a lightweight small language model is preferred. In this work, we finetuned llama-3.2-1b as a relevant grader and achieved a significant increase in precision from 0.1301 to 0.7750. Its precision is comparable to that of llama-3.1-70b. Our code is available at https://github.com/taeheej/Lightweight-Relevance-Grader-in-RAG.