Frustratingly Easy Natural Question Answering
This work addresses the challenge of overcomplicating QA systems for researchers and practitioners by demonstrating that simple methods can achieve strong results on a new benchmark.
The paper tackles the problem of question answering on the Natural Questions benchmark by showing that a simple transfer learning approach from BERT outperforms the previous state-of-the-art by 1.9 F1 points, with ensembling further improving it by 2.3 F1 points.
Existing literature on Question Answering (QA) mostly focuses on algorithmic novelty, data augmentation, or increasingly large pre-trained language models like XLNet and RoBERTa. Additionally, a lot of systems on the QA leaderboards do not have associated research documentation in order to successfully replicate their experiments. In this paper, we outline these algorithmic components such as Attention-over-Attention, coupled with data augmentation and ensembling strategies that have shown to yield state-of-the-art results on benchmark datasets like SQuAD, even achieving super-human performance. Contrary to these prior results, when we evaluate on the recently proposed Natural Questions benchmark dataset, we find that an incredibly simple approach of transfer learning from BERT outperforms the previous state-of-the-art system trained on 4 million more examples than ours by 1.9 F1 points. Adding ensembling strategies further improves that number by 2.3 F1 points.