Latent Prompt Tuning for Text Summarization
This work addresses the need for flexible summarization models that can handle both controlled and uncontrolled scenarios, offering incremental improvements in a domain-specific task.
The paper tackles the problem of text summarization where control signals (e.g., length, keywords) are unavailable at inference time, proposing Lotus, a model that learns latent prompt representations to operate in both controlled and uncontrolled modes, resulting in consistent improvements over strong uncontrollable models across four datasets.
Prompts with different control signals (e.g., length, keywords, etc.) can be used to control text summarization. When control signals are available, they can control the properties of generated summaries and potentially improve summarization quality (since more information are given). Unfortunately, control signals are not already available during inference time. In this paper, we propose Lotus (shorthand for Latent Prompt Tuning for Summarization), which is a single model that can be applied in both controlled and uncontrolled (without control signals) modes. During training, Lotus learns latent prompt representations from prompts with gold control signals using a contrastive learning objective. Experiments show Lotus in uncontrolled mode consistently improves upon strong (uncontrollable) summarization models across four different summarization datasets. We also demonstrate generated summaries can be controlled using prompts with user specified control tokens.