Plans for Evaluating Structured Generative Search Summaries
For search engine developers and researchers, this provides a structured evaluation approach for a new type of AI-generated search result, though it is currently only a plan without results.
The paper proposes a framework for evaluating structured generative search summaries (with overview, sections, and citations) placed atop organic search results, and outlines plans for its implementation and evaluation.
We propose a framework for evaluating structured generative search summaries that are placed atop organic web search results. A structured summary, generated by a large language model, typically consists of an overview, several sections with section titles, and a list of source documents that are cited within the summary. We then describe our plans for implementing and evaluating the framework.