Asking What Matters: Reward-Driven Clarification for Software Engineering Tasks
For developers using AI assistants, this work improves clarification efficiency by reducing unnecessary questions while maintaining task success, addressing a practical bottleneck in human-AI interaction for software engineering.
The paper studies how to generate effective clarifying questions for software engineering tasks by identifying which missing information most impacts task success and which questions users can realistically answer. The proposed CLARITI model matches GPT-5's resolution rate on underspecified issues while generating 41% fewer questions.
Humans often specify tasks incompletely, so assistants must know when and how to ask clarifying questions. However, effective clarification remains challenging in software engineering tasks as not all missing information is equally valuable, and questions must target information users can realistically provide. We study clarification in real software engineering tasks by quantifying which types of information most affect task success and which questions elicit useful responses from simulated users. Using Shapley attribution and distributional comparisons, we identify two key properties of effective clarification: task relevance (which information predicts success) and user answerability (what users can realistically provide). We operationalize these properties as multi-stage reinforcement learning rewards to train CLARITI, an 8B-parameter clarification module, that matches GPT-5's resolution rate on underspecified issues while generating 41% fewer questions. Our results suggest that grounding reward design in empirical analysis of information impact and user answerability improves clarification efficiency.