Optimising Factual Consistency in Summarisation via Preference Learning from Multiple Imperfect Metrics
For practitioners of text summarisation, this work provides a method to enhance factual consistency without relying on a single reliable metric or human annotations, though the gains are incremental over existing approaches.
The paper introduces a training pipeline that improves factual consistency in summarisation by aggregating scores from multiple weak factuality metrics, converting them into preferences, and filtering out high-disagreement cases. Experiments show consistent factuality gains across various model sizes, with smaller models achieving factuality comparable to larger ones.
Reinforcement learning with evaluation metrics as rewards is widely used to enhance specific capabilities of language models. However, for tasks such as factually consistent summarisation, existing metrics remain underdeveloped, limiting their effectiveness as signals for shaping model behaviour.While individual factuality metrics are unreliable, their combination can more effectively capture diverse factual errors. We leverage this insight to introduce an automated training pipeline that improves factual consistency in summaries by aggregating scores from different weak metrics. Our approach avoids the need for complex reward shaping by mapping scores to preferences and filtering out cases with high disagreement between metrics. For each source document, we generate lexically similar summary pairs by varying decoding strategies, enabling the model to learn from factual differences caused by subtle lexical differences. This approach constructs a high-quality preference dataset using only source documents.Experiments demonstrate consistent factuality gains across models, ranging from early encoder-decoder architectures to modern large language models, with smaller models reaching comparable factuality to larger ones.