Discourse-Aware Semantic Self-Attention for Narrative Reading Comprehension
This addresses the challenge of enhancing generalization in self-attention models for reading comprehension tasks, particularly for long texts, though it is incremental as it builds on existing annotation tools and methods.
The paper tackled the problem of reading comprehension on long narrative texts by proposing a discourse-aware semantic self-attention encoder that uses linguistic annotations, resulting in improved performance with specific gains from intra-sentential and cross-sentential discourse relations, semantic role relations, and coreference relations.
In this work, we propose to use linguistic annotations as a basis for a \textit{Discourse-Aware Semantic Self-Attention} encoder that we employ for reading comprehension on long narrative texts. We extract relations between discourse units, events and their arguments as well as coreferring mentions, using available annotation tools. Our empirical evaluation shows that the investigated structures improve the overall performance, especially intra-sentential and cross-sentential discourse relations, sentence-internal semantic role relations, and long-distance coreference relations. We show that dedicating self-attention heads to intra-sentential relations and relations connecting neighboring sentences is beneficial for finding answers to questions in longer contexts. Our findings encourage the use of discourse-semantic annotations to enhance the generalization capacity of self-attention models for reading comprehension.