Inter-Passage Verification for Multi-evidence Multi-answer QA
This addresses the problem of multi-evidence multi-answer QA for retrieval-augmented generation systems, offering a novel method to improve answer synthesis.
The paper tackles the challenge of multi-answer question answering, where systems struggle to retrieve and synthesize many evidence passages, by proposing a new framework called RI$^2$VER that processes passages individually and uses inter-passage verification to validate answers, resulting in an average F1 score improvement of 11.17% on datasets like QAMPARI and RoMQA.
Multi-answer question answering (QA), where questions can have many valid answers, presents a significant challenge for existing retrieval-augmented generation-based QA systems, as these systems struggle to retrieve and then synthesize a large number of evidence passages. To tackle these challenges, we propose a new multi-answer QA framework -- Retrieval-augmented Independent Reading with Inter-passage Verification (RI$^2$VER). Our framework retrieves a large set of passages and processes each passage individually to generate an initial high-recall but noisy answer set. Then we propose a new inter-passage verification pipeline that validates every candidate answer through (1) Verification Question Generation, (2) Gathering Additional Evidence, and (3) Verification with inter-passage synthesis. Evaluations on the QAMPARI and RoMQA datasets demonstrate that our framework significantly outperforms existing baselines across various model sizes, achieving an average F1 score improvement of 11.17%. Further analysis validates that our inter-passage verification pipeline enables our framework to be particularly beneficial for questions requiring multi-evidence synthesis.