Graph Sequential Network for Reasoning over Sequences
It addresses reasoning over sequences in NLP tasks like reading comprehension and fact verification, offering an incremental improvement over existing GNN approaches.
The paper tackles reasoning over graphs built from sequences by proposing Graph Sequential Network (GSN), a new GNN with a co-attention message passing algorithm, which achieves better performance than standard GNN methods on HotpotQA and FEVER tasks.
Recently Graph Neural Network (GNN) has been applied successfully to various NLP tasks that require reasoning, such as multi-hop machine reading comprehension. In this paper, we consider a novel case where reasoning is needed over graphs built from sequences, i.e. graph nodes with sequence data. Existing GNN models fulfill this goal by first summarizing the node sequences into fixed-dimensional vectors, then applying GNN on these vectors. To avoid information loss inherent in the early summarization and make sequential labeling tasks on GNN output feasible, we propose a new type of GNN called Graph Sequential Network (GSN), which features a new message passing algorithm based on co-attention between a node and each of its neighbors. We validate the proposed GSN on two NLP tasks: interpretable multi-hop reading comprehension on HotpotQA and graph based fact verification on FEVER. Both tasks require reasoning over multiple documents or sentences. Our experimental results show that the proposed GSN attains better performance than the standard GNN based methods.