Answer Generation for Retrieval-based Question Answering Systems
This addresses a key limitation in retrieval-based QA systems for users needing reliable answers, though it is an incremental improvement over existing transformer-based approaches.
The paper tackles the problem of poor-quality retrieved candidates in answer sentence selection (AS2) models for question answering by generating answers from top candidates instead of selecting the best one, achieving up to 32 absolute points improvement in accuracy over state-of-the-art methods on three datasets.
Recent advancements in transformer-based models have greatly improved the ability of Question Answering (QA) systems to provide correct answers; in particular, answer sentence selection (AS2) models, core components of retrieval-based systems, have achieved impressive results. While generally effective, these models fail to provide a satisfying answer when all retrieved candidates are of poor quality, even if they contain correct information. In AS2, models are trained to select the best answer sentence among a set of candidates retrieved for a given question. In this work, we propose to generate answers from a set of AS2 top candidates. Rather than selecting the best candidate, we train a sequence to sequence transformer model to generate an answer from a candidate set. Our tests on three English AS2 datasets show improvement up to 32 absolute points in accuracy over the state of the art.