HalluClear: Diagnosing, Evaluating and Mitigating Hallucinations in GUI Agents
For GUI agent developers, HalluClear provides a targeted solution to the critical problem of hallucinations, which cause cascading failures in real-world deployments.
The paper introduces HalluClear, a suite for diagnosing, evaluating, and mitigating hallucinations in GUI agents. Using only 9K post-training samples, it significantly reduces hallucinations and improves grounding and action fidelity, offering a compute-efficient alternative to scaling.
While progress in GUI agents has been largely driven by industrial-scale training, ungrounded hallucinations often trigger cascading failures in real-world deployments.Unlike general VLM domains, the GUI agent field lacks a hallucination-focused suite for fine-grained diagnosis, reliable evaluation, and targeted mitigation.To bridge this gap, we introduce HalluClear, a comprehensive suite for hallucination mitigation in GUI agents as a complement to computation-intensive scaling. HalluClear comprises: (1) a GUI-specific hallucination taxonomy derived from empirical failure analysis; (2) a calibrated three-stage evaluation workflow which enhances VLM-as-a-judge reliability via expert-annotated benchmarking and ensemble credibility estimation; and (3) a mitigation scheme based on closed-loop structured reasoning, enabling lightweight continual post-training with cold-start initialization for both generalist and GUI-specialist agents. Experiments across representative agents and public benchmarks demonstrate that post-training on only 9K samples within our suite can significantly reduce hallucinations, thereby improving grounding and action fidelity, offering a compute-efficient pathway to robust GUI automation.