Text Summarization with Oracle Expectation
This work addresses a key bottleneck in extractive summarization training, offering an incremental improvement over existing labeling methods.
The paper tackled the problem of suboptimal and deterministic oracle labels in extractive summarization by proposing a soft, expectation-based labeling algorithm, achieving superior performance on multiple benchmarks across domains and languages.
Extractive summarization produces summaries by identifying and concatenating the most important sentences in a document. Since most summarization datasets do not come with gold labels indicating whether document sentences are summary-worthy, different labeling algorithms have been proposed to extrapolate oracle extracts for model training. In this work, we identify two flaws with the widely used greedy labeling approach: it delivers suboptimal and deterministic oracles. To alleviate both issues, we propose a simple yet effective labeling algorithm that creates soft, expectation-based sentence labels. We define a new learning objective for extractive summarization which incorporates learning signals from multiple oracle summaries and prove it is equivalent to estimating the oracle expectation for each document sentence. Without any architectural modifications, the proposed labeling scheme achieves superior performance on a variety of summarization benchmarks across domains and languages, in both supervised and zero-shot settings.