Learning to Control Summaries with Score Ranking
For summarization researchers and practitioners, this work provides a method to control individual quality dimensions in generated summaries, addressing a gap in existing multi-criteria optimization approaches.
The paper addresses the challenge of controlling summary generation across individual quality criteria (e.g., completeness, conciseness, faithfulness) despite their trade-offs. It proposes a loss function that aligns model outputs with fine-grained evaluation scores, achieving performance comparable to state-of-the-art summarizers while offering strong controllability over specific dimensions.
Recent advances in summarization research focus on improving summary quality across multiple criteria, such as completeness, conciseness, and faithfulness, by jointly optimizing these dimensions. However, these efforts largely overlook the challenge of controlling summary generation with respect to individual criteria, especially in the presence of their inherent trade-offs. For example, enhancing conciseness can compromise completeness, and vice versa. In this work, we address this gap by proposing a loss function that aligns model outputs with fine-grained, model-based evaluation scores (e.g., from FineSurE), enabling both improvement in summary quality and dimension-specific control. Our approach improves the overall quality of summaries while maintaining the ability to selectively prioritize one criterion over others. Experiments on three pretrained models (LLaMA, Qwen, and Mistral) demonstrate that our method achieves performance comparable to state-of-the-art summarizers, while uniquely offering strong controllability over individual quality dimensions.