Just What You Desire: Constrained Timeline Summarization with Self-Reflection for Enhanced Relevance
This work addresses the problem of personalized and relevant timeline generation for users in news summarization, though it is incremental as it builds on existing timeline summarization tasks with added constraints.
The paper tackles the underspecified nature of timeline summarization by introducing Constrained Timeline Summarization (CTLS), where timelines are generated to meet specific constraints, such as focusing on legal battles for an entity like Tiger Woods, and proposes an LLM-based approach with self-reflection that improves performance on a new dataset of 47 entities with 5 constraints each.
Given news articles about an entity, such as a public figure or organization, timeline summarization (TLS) involves generating a timeline that summarizes the key events about the entity. However, the TLS task is too underspecified, since what is of interest to each reader may vary, and hence there is not a single ideal or optimal timeline. In this paper, we introduce a novel task, called Constrained Timeline Summarization (CTLS), where a timeline is generated in which all events in the timeline meet some constraint. An example of a constrained timeline concerns the legal battles of Tiger Woods, where only events related to his legal problems are selected to appear in the timeline. We collected a new human-verified dataset of constrained timelines involving 47 entities and 5 constraints per entity. We propose an approach that employs a large language model (LLM) to summarize news articles according to a specified constraint and cluster them to identify key events to include in a constrained timeline. In addition, we propose a novel self-reflection method during summary generation, demonstrating that this approach successfully leads to improved performance.