No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension
This work addresses the problem of improving machine reading comprehension for researchers and practitioners by handling varied answer types, though it is incremental as it builds on existing benchmarks and methods.
The paper tackled the challenge of handling diverse answer types in the Natural Questions benchmark by proposing a two-step training approach to identify no-answer and wrong-answer cases, achieving top performance with F1 scores of 77.2 for long answers and 64.1 for short answers.
The Natural Questions (NQ) benchmark set brings new challenges to Machine Reading Comprehension: the answers are not only at different levels of granularity (long and short), but also of richer types (including no-answer, yes/no, single-span and multi-span). In this paper, we target at this challenge and handle all answer types systematically. In particular, we propose a novel approach called Reflection Net which leverages a two-step training procedure to identify the no-answer and wrong-answer cases. Extensive experiments are conducted to verify the effectiveness of our approach. At the time of paper writing (May.~20,~2020), our approach achieved the top 1 on both long and short answer leaderboard, with F1 scores of 77.2 and 64.1, respectively.