KLearn: Background Knowledge Inference from Summarization Data
This work addresses the understudied aspect of background knowledge in summarization, offering a framework with potential applications in improving summary evaluation and understanding human information priors.
The paper tackles the problem of inferring background knowledge from summarization data, showing that scoring functions modeling background knowledge fit human judgments significantly better than baselines.
The goal of text summarization is to compress documents to the relevant information while excluding background information already known to the receiver. So far, summarization researchers have given considerably more attention to relevance than to background knowledge. In contrast, this work puts background knowledge in the foreground. Building on the realization that the choices made by human summarizers and annotators contain implicit information about their background knowledge, we develop and compare techniques for inferring background knowledge from summarization data. Based on this framework, we define summary scoring functions that explicitly model background knowledge, and show that these scoring functions fit human judgments significantly better than baselines. We illustrate some of the many potential applications of our framework. First, we provide insights into human information importance priors. Second, we demonstrate that averaging the background knowledge of multiple, potentially biased annotators or corpora greatly improves summary-scoring performance. Finally, we discuss potential applications of our framework beyond summarization.