From Verification Burden to Trusted Collaboration: Design Goals for LLM-Assisted Literature Reviews
This addresses the problem of inefficient and untrustworthy LLM-assisted literature reviews for researchers, though it is incremental as it builds on existing practices with a new framework.
The paper tackled the problem of underexplored implementation and design challenges of LLMs in literature reviews by conducting a user study to identify gaps like lack of trust and verification burden, resulting in a proposed framework with six design goals to improve trust and collaboration.
Large Language Models (LLMs) are increasingly embedded in academic writing practices. Although numerous studies have explored how researchers employ these tools for scientific writing, their concrete implementation, limitations, and design challenges within the literature review process remain underexplored. In this paper, we report a user study with researchers across multiple disciplines to characterize current practices, benefits, and \textit{pain points} in using LLMs to investigate related work. We identified three recurring gaps: (i) lack of trust in outputs, (ii) persistent verification burden, and (iii) requiring multiple tools. This motivates our proposal of six design goals and a high-level framework that operationalizes them through improved related papers visualization, verification at every step, and human-feedback alignment with generation-guided explanations. Overall, by grounding our work in the practical, day-to-day needs of researchers, we designed a framework that addresses these limitations and models real-world LLM-assisted writing, advancing trust through verifiable actions and fostering practical collaboration between researchers and AI systems.