Abstractive Summarization Using Attentive Neural Techniques
This work addresses the problem of automatic text summarization for handling proliferating data, but it is incremental as it applies existing techniques from machine translation to a related domain.
The paper tackled abstractive text summarization by adapting attention-based encoder-decoder networks from machine translation, modifying and optimizing a translation model with self-attention to generate sentence summaries, and analyzed its effectiveness on standardized evaluation sets while noting metric limitations.
In a world of proliferating data, the ability to rapidly summarize text is growing in importance. Automatic summarization of text can be thought of as a sequence to sequence problem. Another area of natural language processing that solves a sequence to sequence problem is machine translation, which is rapidly evolving due to the development of attention-based encoder-decoder networks. This work applies these modern techniques to abstractive summarization. We perform analysis on various attention mechanisms for summarization with the goal of developing an approach and architecture aimed at improving the state of the art. In particular, we modify and optimize a translation model with self-attention for generating abstractive sentence summaries. The effectiveness of this base model along with attention variants is compared and analyzed in the context of standardized evaluation sets and test metrics. However, we show that these metrics are limited in their ability to effectively score abstractive summaries, and propose a new approach based on the intuition that an abstractive model requires an abstractive evaluation.