Controlling Summarization Length Through EOS Token Weighting
This provides a simple, compatible solution for users needing length control in summarization, though it is incremental as it builds on existing token prediction methods.
The paper tackled the problem of controlling summary length in text generation by adjusting the EOS token weighting in loss computation, resulting in a method that works with various models and maintains summary quality.
Controlling the length of generated text can be crucial in various text-generation tasks, including summarization. Existing methods often require complex model alterations, limiting compatibility with pre-trained models. We address these limitations by developing a simple approach for controlling the length of automatic text summaries by increasing the importance of correctly predicting the EOS token in the cross-entropy loss computation. The proposed methodology is agnostic to architecture and decoding algorithms and orthogonal to other inference-time techniques to control the generation length. We tested it with encoder-decoder and modern GPT-style LLMs, and show that this method can control generation length, often without affecting the quality of the summary.