Summarization of Opinionated Political Documents with Varied Perspectives
This addresses polarization reduction for users of political news, but it is incremental as it focuses on a specific dataset and evaluation framework.
The paper tackled the problem of summarizing opinionated political documents from varied perspectives to reduce polarization, finding that all tested models, including GPT-4o, struggle to generate summaries faithful to the intended perspectives.
Global partisan hostility and polarization has increased, and this polarization is heightened around presidential elections. Models capable of generating accurate summaries of diverse perspectives can help reduce such polarization by exposing users to alternative perspectives. In this work, we introduce a novel dataset and task for independently summarizing each political perspective in a set of passages from opinionated news articles. For this task, we propose a framework for evaluating different dimensions of perspective summary performance. We benchmark 11 summarization models and LLMs of varying sizes and architectures through both automatic and human evaluation. While recent models like GPT-4o perform well on this task, we find that all models struggle to generate summaries that are faithful to the intended perspective. Our analysis of summaries focuses on how extraction behavior is impacted by features of the input documents.