MS-Ranker: Accumulating Evidence from Potentially Correct Candidates for Answer Selection
This work addresses answer selection for question-answering systems, offering a novel method that improves accuracy without external resources, though it is incremental in building upon existing ranking approaches.
The paper tackles the problem of answer selection by addressing the lack of matching information between questions and candidate answers, proposing MS-Ranker, a reinforcement learning-based multi-step ranking model that accumulates evidence from potentially correct candidates, resulting in significant performance improvements on benchmarks like WikiQA and SemEval-2016 CQA.
As conventional answer selection (AS) methods generally match the question with each candidate answer independently, they suffer from the lack of matching information between the question and the candidate. To address this problem, we propose a novel reinforcement learning (RL) based multi-step ranking model, named MS-Ranker, which accumulates information from potentially correct candidate answers as extra evidence for matching the question with a candidate. In specific, we explicitly consider the potential correctness of candidates and update the evidence with a gating mechanism. Moreover, as we use a listwise ranking reward, our model learns to pay more attention to the overall performance. Experiments on two benchmarks, namely WikiQA and SemEval-2016 CQA, show that our model significantly outperforms existing methods that do not rely on external resources.