Decision-Focused Summarization
This addresses the problem of irrelevant information in summaries for decision-making tasks, such as medical risk analysis or business predictions, though it is incremental as it builds on existing summarization and explanation methods.
The paper tackles the problem of generating summaries that are relevant to a specific decision, rather than just textually relevant, by proposing decision-focused summarization and a method (DecSum) that selects sentences leading to similar model decisions as the full text. The result shows DecSum substantially outperforms text-only and model-based methods in decision faithfulness and representativeness, and enables humans to predict better-rated restaurants above random chance.
Relevance in summarization is typically defined based on textual information alone, without incorporating insights about a particular decision. As a result, to support risk analysis of pancreatic cancer, summaries of medical notes may include irrelevant information such as a knee injury. We propose a novel problem, decision-focused summarization, where the goal is to summarize relevant information for a decision. We leverage a predictive model that makes the decision based on the full text to provide valuable insights on how a decision can be inferred from text. To build a summary, we then select representative sentences that lead to similar model decisions as using the full text while accounting for textual non-redundancy. To evaluate our method (DecSum), we build a testbed where the task is to summarize the first ten reviews of a restaurant in support of predicting its future rating on Yelp. DecSum substantially outperforms text-only summarization methods and model-based explanation methods in decision faithfulness and representativeness. We further demonstrate that DecSum is the only method that enables humans to outperform random chance in predicting which restaurant will be better rated in the future.