Learning Structured Text Representations
This addresses the challenge of capturing document structure for NLP applications, offering an incremental improvement by integrating parsing algorithms into neural models.
The paper tackles the problem of learning structure-aware document representations without needing a discourse parser or extra annotations, achieving state-of-the-art results on document modeling tasks.
In this paper, we focus on learning structure-aware document representations from data without recourse to a discourse parser or additional annotations. Drawing inspiration from recent efforts to empower neural networks with a structural bias, we propose a model that can encode a document while automatically inducing rich structural dependencies. Specifically, we embed a differentiable non-projective parsing algorithm into a neural model and use attention mechanisms to incorporate the structural biases. Experimental evaluation across different tasks and datasets shows that the proposed model achieves state-of-the-art results on document modeling tasks while inducing intermediate structures which are both interpretable and meaningful.