Does This Summary Answer My Question? Modeling Query-Focused Summary Readers with Rational Speech Acts
This addresses the user-oriented nature of QFS by explicitly modeling reader understanding, though it is incremental as it builds on existing QFS systems.
The paper tackles the problem of query-focused summarization (QFS) systems underperforming by not considering user understanding, and it introduces an answer reconstruction objective based on the Rational Speech Act framework to re-rank summaries, improving alignment with queries and reference summaries.
Query-focused summarization (QFS) is the task of generating a summary in response to a user-written query. Despite its user-oriented nature, there has been limited work in QFS in explicitly considering a user's understanding of a generated summary, potentially causing QFS systems to underperform at inference time. In this paper, we adapt the Rational Speech Act (RSA) framework, a model of human communication, to explicitly model a reader's understanding of a query-focused summary and integrate it within the generation method of existing QFS systems. In particular, we introduce the answer reconstruction objective which approximates a reader's understanding of a summary by their ability to use it to reconstruct the answer to their initial query. Using this objective, we are able to re-rank candidate summaries generated by existing QFS systems and select summaries that better align with their corresponding query and reference summary. More generally, our study suggests that a simple and effective way of improving a language generation system designed for a user-centered task may be to explicitly incorporate its user requirements into the system's generation procedure.