Topic Augmented Generator for Abstractive Summarization
This work addresses the problem of generating more coherent summaries for users in NLP applications, but it is incremental as it builds on existing sequence-to-sequence models.
The paper tackles abstractive summarization by proposing a decoder that conditions on both input text and latent topics, achieving strongly improved ROUGE scores on CNN/Daily Mail and WikiHow datasets compared to state-of-the-art models.
Steady progress has been made in abstractive summarization with attention-based sequence-to-sequence learning models. In this paper, we propose a new decoder where the output summary is generated by conditioning on both the input text and the latent topics of the document. The latent topics, identified by a topic model such as LDA, reveals more global semantic information that can be used to bias the decoder to generate words. In particular, they enable the decoder to have access to additional word co-occurrence statistics captured at document corpus level. We empirically validate the advantage of the proposed approach on both the CNN/Daily Mail and the WikiHow datasets. Concretely, we attain strongly improved ROUGE scores when compared to state-of-the-art models.