Faithful Summarisation under Disagreement via Belief-Level Aggregation
This addresses the issue of over-representing majority opinions in opinion and multi-document summarization, which is important for applications requiring balanced and accurate synthesis of conflicting information.
The paper tackles the problem of generating faithful summaries when documents contain conflicting viewpoints, by separating belief-level aggregation from language generation, resulting in consistently strong disagreement-aware performance across different model architectures and capacities.
Opinion and multi-document summarisation often involve genuinely conflicting viewpoints, yet many existing approaches, particularly LLM-based systems, implicitly smooth disagreement and over-represent majority opinions. This limits the faithfulness of generated summaries in opinion-heavy settings. We introduce a disagreement-aware synthesis pipeline that separates belief-level aggregation from language generation. Documents are first represented as structured belief sets and aggregated using distance-based belief merging operators that explicitly model conflict. Large language models are then used only to realise the aggregated beliefs as natural language summaries. We evaluate the approach across multiple model families and scales, comparing it to methods that perform explicit aggregation during generation. Our results show that while sufficiently large models can match belief-level aggregation when aggregation is handled at generation time, this behaviour is not stable across architectures or capacities. In contrast, belief-level aggregation combined with simple prompting yields consistently strong disagreement-aware performance across models, while maintaining fluent and grounded summaries.