Improved Document Modelling with a Neural Discourse Parser
This work addresses a bottleneck in NLP for tasks involving longer documents, offering an incremental improvement over existing methods.
The paper tackled the problem of neural models' inability to capture discourse structure in larger documents by using neural discourse representations from an RST parser to enhance document representations, resulting in improvements on abstractive summarization and popularity prediction tasks.
Despite the success of attention-based neural models for natural language generation and classification tasks, they are unable to capture the discourse structure of larger documents. We hypothesize that explicit discourse representations have utility for NLP tasks over longer documents or document sequences, which sequence-to-sequence models are unable to capture. For abstractive summarization, for instance, conventional neural models simply match source documents and the summary in a latent space without explicit representation of text structure or relations. In this paper, we propose to use neural discourse representations obtained from a rhetorical structure theory (RST) parser to enhance document representations. Specifically, document representations are generated for discourse spans, known as the elementary discourse units (EDUs). We empirically investigate the benefit of the proposed approach on two different tasks: abstractive summarization and popularity prediction of online petitions. We find that the proposed approach leads to improvements in all cases.