Probabilistic Assumptions Matter: Improved Models for Distantly-Supervised Document-Level Question Answering
This improves document-level question answering for applications relying on distant supervision, though it is incremental as it refines existing probabilistic assumptions rather than introducing a new paradigm.
The paper tackles the problem of extractive question answering using document-level distant supervision by analyzing how different probabilistic assumptions about weak answer string labels interact. The result is a multi-objective model that outperforms previous state-of-the-art by 4.3 F1 points on TriviaQA-Wiki and 1.7 Rouge-L points on NarrativeQA summaries.
We address the problem of extractive question answering using document-level distant super-vision, pairing questions and relevant documents with answer strings. We compare previously used probability space and distant super-vision assumptions (assumptions on the correspondence between the weak answer string labels and possible answer mention spans). We show that these assumptions interact, and that different configurations provide complementary benefits. We demonstrate that a multi-objective model can efficiently combine the advantages of multiple assumptions and out-perform the best individual formulation. Our approach outperforms previous state-of-the-art models by 4.3 points in F1 on TriviaQA-Wiki and 1.7 points in Rouge-L on NarrativeQA summaries.