Hierarchical Transformers for Multi-Document Summarization
This addresses the problem of generating coherent summaries from multiple documents for applications like news aggregation or research synthesis, representing an incremental advance in neural summarization methods.
The paper tackles multi-document summarization by developing a hierarchical Transformer model that encodes cross-document relationships through attention mechanisms, achieving substantial improvements over strong baselines on the WikiSum dataset.
In this paper, we develop a neural summarization model which can effectively process multiple input documents and distill Transformer architecture with the ability to encode documents in a hierarchical manner. We represent cross-document relationships via an attention mechanism which allows to share information as opposed to simply concatenating text spans and processing them as a flat sequence. Our model learns latent dependencies among textual units, but can also take advantage of explicit graph representations focusing on similarity or discourse relations. Empirical results on the WikiSum dataset demonstrate that the proposed architecture brings substantial improvements over several strong baselines.