Automatic Text Extractive Summarization Based on Graph and Pre-trained Language Model Attention
This work addresses extractive summarization for NLP applications, but it is incremental as it builds on existing graph and attention methods.
The paper tackles automatic text summarization by combining graph representations with pre-trained language model attention, achieving competitive results on two typical datasets compared to state-of-the-art models.
Representing a text as a graph for obtaining automatic text summarization has been investigated for over ten years. With the development of attention or Transformer on natural language processing (NLP), it is possible to make a connection between the graph and attention structure for a text. In this paper, an attention matrix between the sentences of the whole text is adopted as a weighted adjacent matrix of a fully connected graph of the text, which can be produced through the pre-training language model. The GCN is further applied to the text graph model for classifying each node and finding out the salient sentences from the text. It is demonstrated by the experimental results on two typical datasets that our proposed model can achieve a competitive result in comparison with sate-of-the-art models.