Don't Blindly Trust It: How Unreliable Feedback Breaks Tool-Using LLM Agents
For developers of tool-augmented agents, this work highlights that clean-tool gains can overstate tool value and that no-feedback fallback controls are necessary for evaluation.
The paper shows that when tool-augmented LLM agents receive unreliable feedback, they can perform worse than a no-feedback baseline, with Qwen2.5-7B dropping from 44.8 F1 (clean retrieval) to 4.7 F1 (shuffled retrieval) on HotpotQA, demonstrating a value inversion.
Tool-augmented agents are typically evaluated by their gains under reliable external feedback. Yet these gains leave open a key counterfactual: when feedback is unreliable, would the agent be better off receiving no task evidence? We study this question with a controlled matched-loop comparison that fixes the agent loop, prompt, action space, and decoding, while varying only the returned observation: faithful, misleading, or absent. Across question answering and fact verification, persistent misleading feedback produces a value inversion: agents that benefit from clean tools can perform worse than the matched no-feedback fallback. On HotpotQA, Qwen2.5-7B reaches 44.8 F1 with clean retrieval and 22.3 F1 with no feedback, but drops to 4.7 F1 under shuffled retrieval. The inversion persists under stronger clean retrieval and locally plausible distractors, but weakens when later clean evidence can repair the trajectory. Early trajectory signals predict many failures, yet simple repairs remain fallback-limited: rejecting bad evidence helps only when the exposed fallback is reliable. These results show that clean-tool gains can overstate tool value, and that matched no-feedback fallback controls are necessary for evaluating tool-augmented agents.