WikiPassageQA: A Benchmark Collection for Research on Non-factoid Answer Passage Retrieval
This provides a benchmark for researchers working on non-factoid question answering in information retrieval, though it is incremental as it addresses a data gap rather than a methodological breakthrough.
The authors tackled the lack of datasets for non-factoid answer passage retrieval by introducing WikiPassageQA, a Wikipedia-based collection with thousands of annotated questions, and benchmarked it on various neural and retrieval models to highlight unique challenges.
With the rise in mobile and voice search, answer passage retrieval acts as a critical component of an effective information retrieval system for open domain question answering. Currently, there are no comparable collections that address non-factoid question answering within larger documents while simultaneously providing enough examples sufficient to train a deep neural network. In this paper, we introduce a new Wikipedia based collection specific for non-factoid answer passage retrieval containing thousands of questions with annotated answers and show benchmark results on a variety of state of the art neural architectures and retrieval models. The experimental results demonstrate the unique challenges presented by answer passage retrieval within topically relevant documents for future research.