A Survey of Machine Narrative Reading Comprehension Assessments
This work provides a framework for researchers in natural language processing to better evaluate and design narrative comprehension tasks, though it is incremental as it synthesizes existing knowledge rather than introducing new methods.
The paper addresses the need for systematic assessment strategies in machine narrative comprehension by proposing a typology based on narrative theories, reading comprehension theories, and existing tasks and datasets, and discusses implications for task design and challenges.
As the body of research on machine narrative comprehension grows, there is a critical need for consideration of performance assessment strategies as well as the depth and scope of different benchmark tasks. Based on narrative theories, reading comprehension theories, as well as existing machine narrative reading comprehension tasks and datasets, we propose a typology that captures the main similarities and differences among assessment tasks; and discuss the implications of our typology for new task design and the challenges of narrative reading comprehension.