Frustratingly Hard Evidence Retrieval for QA Over Books
This addresses the challenge of evidence retrieval for QA over books, which is an incremental improvement in a domain-specific area.
The paper tackles the problem of question answering over narrative books by formulating it as an open-domain QA task and investigates state-of-the-art approaches, achieving state-of-the-art results on the NarrativeQA benchmark while revealing the difficulty of evidence retrieval in books through experiments and analysis.
A lot of progress has been made to improve question answering (QA) in recent years, but the special problem of QA over narrative book stories has not been explored in-depth. We formulate BookQA as an open-domain QA task given its similar dependency on evidence retrieval. We further investigate how state-of-the-art open-domain QA approaches can help BookQA. Besides achieving state-of-the-art on the NarrativeQA benchmark, our study also reveals the difficulty of evidence retrieval in books with a wealth of experiments and analysis - which necessitates future effort on novel solutions for evidence retrieval in BookQA.