BanditSum: Extractive Summarization as a Contextual Bandit
This addresses the challenge of efficient and effective extractive summarization for NLP applications, though it is incremental in its method adaptation.
The paper tackles the problem of training neural networks for extractive summarization without heuristic labels by framing it as a contextual bandit problem, achieving ROUGE scores comparable to state-of-the-art methods and converging with fewer update steps.
In this work, we propose a novel method for training neural networks to perform single-document extractive summarization without heuristically-generated extractive labels. We call our approach BanditSum as it treats extractive summarization as a contextual bandit (CB) problem, where the model receives a document to summarize (the context), and chooses a sequence of sentences to include in the summary (the action). A policy gradient reinforcement learning algorithm is used to train the model to select sequences of sentences that maximize ROUGE score. We perform a series of experiments demonstrating that BanditSum is able to achieve ROUGE scores that are better than or comparable to the state-of-the-art for extractive summarization, and converges using significantly fewer update steps than competing approaches. In addition, we show empirically that BanditSum performs significantly better than competing approaches when good summary sentences appear late in the source document.