Towards Inference-Oriented Reading Comprehension: ParallelQA
This addresses a fundamental limitation in reading comprehension systems for AI research, though it is incremental as it focuses on a specific reasoning type.
The paper tackles the problem of neural Machine Reading Comprehension models relying on shallow pattern matching instead of inference-oriented reasoning, and demonstrates that existing models fail to generalize to their proposed ParallelQA benchmark for referential inference questions.
In this paper, we investigate the tendency of end-to-end neural Machine Reading Comprehension (MRC) models to match shallow patterns rather than perform inference-oriented reasoning on RC benchmarks. We aim to test the ability of these systems to answer questions which focus on referential inference. We propose ParallelQA, a strategy to formulate such questions using parallel passages. We also demonstrate that existing neural models fail to generalize well to this setting.