Attribution Gradients: Incrementally Unfolding Citations for Critical Examination of Attributed AI Answers
This addresses the challenge of verifying attributed AI answers for users, though it is incremental as it builds on existing attribution methods with new UI features.
The paper tackles the problem of verifying content in AI-generated answers with attributions by introducing attribution gradients, which allow users to decompose answers into claims and view supporting/contradictory excerpts from sources, leading to greater engagement and richer revisions in a usability study.
AI question answering systems increasingly generate responses with attributions to sources. However, the task of verifying the actual content of these attributions is in most cases impractical. In this paper, we present attribution gradients as a solution. Attribution gradients provide integrated, incremental affordances for diving into an attributed passage. A user can decompose a sentence of an answer into its claims. For each claim, the user can view supporting and contradictory excerpts mined from sources. Those excerpts serve as clickable conduits into the source (in our application, scientific papers). When evidence itself contains more citations, the UI unpacks the evidence into excerpts from the cited sources. These features of attribution gradients facilitate concurrent interconnections among answer, claim, excerpt, and context. In a usability study, we observed greater engagement with sources and richer revision in a task where participants revised an attributed AI answer with attribution gradients and a baseline.