Divide & Conquer for Entailment-aware Multi-hop Evidence Retrieval
This work addresses multi-hop question answering by improving evidence retrieval, offering a domain-specific advancement for QA systems.
The paper tackled the problem of multi-hop evidence retrieval by incorporating textual entailment as a relevance signal, proposing ensemble models that significantly outperformed single retrieval models and intuitive baselines on HotpotQA.
Lexical and semantic matches are commonly used as relevance measurements for information retrieval. Together they estimate the semantic equivalence between the query and the candidates. However, semantic equivalence is not the only relevance signal that needs to be considered when retrieving evidences for multi-hop questions. In this work, we demonstrate that textual entailment relation is another important relevance dimension that should be considered. To retrieve evidences that are either semantically equivalent to or entailed by the question simultaneously, we divide the task of evidence retrieval for multi-hop question answering (QA) into two sub-tasks, i.e., semantic textual similarity and inference similarity retrieval. We propose two ensemble models, EAR and EARnest, which tackle each of the sub-tasks separately and then jointly re-rank sentences with the consideration of the diverse relevance signals. Experimental results on HotpotQA verify that our models not only significantly outperform all the single retrieval models it is based on, but is also more effective than two intuitive ensemble baseline models.