RECAP: REwriting Conversations for Intent Understanding in Agentic Planning
This addresses the challenge of intent detection for open-domain dialogue systems, though it is incremental as it builds on existing LLM-based methods.
The authors tackled the problem of ambiguous user intent in conversational assistants by creating RECAP, a benchmark for evaluating intent rewriting approaches, and developed a prompt-based method that outperformed baselines, with fine-tuned rewriters yielding additional gains.
Understanding user intent is essential for effective planning in conversational assistants, particularly those powered by large language models (LLMs) coordinating multiple agents. However, real-world dialogues are often ambiguous, underspecified, or dynamic, making intent detection a persistent challenge. Traditional classification-based approaches struggle to generalize in open-ended settings, leading to brittle interpretations and poor downstream planning. We propose RECAP (REwriting Conversations for Agent Planning), a new benchmark designed to evaluate and advance intent rewriting, reframing user-agent dialogues into concise representations of user goals. RECAP captures diverse challenges such as ambiguity, intent drift, vagueness, and mixed-goal conversations. Alongside the dataset, we introduce an LLM-based evaluator that assesses planning utility given the rewritten intent. Using RECAP, we develop a prompt-based rewriting approach that outperforms baselines. We further demonstrate that fine-tuning two DPO-based rewriters yields additional utility gains. Our results highlight intent rewriting as a critical and tractable component for improving agent planning in open-domain dialogue systems.