Rag Performance Prediction for Question Answering
This work addresses the problem of optimizing RAG systems for question answering, but it is incremental as it builds on existing predictors with a new supervised approach.
The paper tackled predicting the performance gain of using retrieval-augmented generation (RAG) for question answering compared to not using it, finding that a novel supervised predictor modeling semantic relationships among question, passages, and answer achieved the best prediction quality.
We address the task of predicting the gain of using RAG (retrieval augmented generation) for question answering with respect to not using it. We study the performance of a few pre-retrieval and post-retrieval predictors originally devised for ad hoc retrieval. We also study a few post-generation predictors, one of which is novel to this study and posts the best prediction quality. Our results show that the most effective prediction approach is a novel supervised predictor that explicitly models the semantic relationships among the question, retrieved passages, and the generated answer.