Self-Supervised Test-Time Learning for Reading Comprehension
This addresses the problem of reducing reliance on large-scale human-authored datasets for reading comprehension, though it is incremental as it builds on existing unsupervised question-answering techniques.
The paper tackles unsupervised reading comprehension by introducing a test-time learning method that trains models on synthetically generated question-answer pairs from a single test context, achieving accuracies competitive with fully supervised methods and significantly outperforming current unsupervised approaches.
Recent work on unsupervised question answering has shown that models can be trained with procedurally generated question-answer pairs and can achieve performance competitive with supervised methods. In this work, we consider the task of unsupervised reading comprehension and present a method that performs "test-time learning" (TTL) on a given context (text passage), without requiring training on large-scale human-authored datasets containing \textit{context-question-answer} triplets. This method operates directly on a single test context, uses self-supervision to train models on synthetically generated question-answer pairs, and then infers answers to unseen human-authored questions for this context. Our method achieves accuracies competitive with fully supervised methods and significantly outperforms current unsupervised methods. TTL methods with a smaller model are also competitive with the current state-of-the-art in unsupervised reading comprehension.