DiSCo: Making Absence Visible in Intelligent Summarization Interfaces
This addresses the problem of misleading summaries for users relying on intelligent interfaces in domains like accommodations, offering an incremental improvement by reducing presence bias to enhance transparency and decision support.
The paper tackled the problem of presence bias in LLM-generated summaries, which overlook missing information and can mislead users, by introducing DiSCo, an expectation-based approach that identifies and integrates unusually emphasized or missing aspects relative to domain norms. In a user study across three accommodation domains, DiSCo summaries were rated as more detailed and useful for decision-making than baseline LLM summaries, although slightly harder to read.
Intelligent interfaces increasingly use large language models to summarize user-generated content, yet these summaries emphasize what is mentioned while overlooking what is missing. This presence bias can mislead users who rely on summaries to make decisions. We present Domain Informed Summarization through Contrast (DiSCo), an expectation-based computational approach that makes absences visible by comparing each entity's content with domain topical expectations captured in reference distributions of aspects typically discussed in comparable accommodations. This comparison identifies aspects that are either unusually emphasized or missing relative to domain norms and integrates them into the generated text. In a user study across three accommodation domains, namely ski, beach, and city center, DiSCo summaries were rated as more detailed and useful for decision making than baseline large language model summaries, although slightly harder to read. The findings show that modeling expectations reduces presence bias and improves both transparency and decision support in intelligent summarization interfaces.