LGJul 20

SEE: Structure-aware Exploring \& Exploiting for Long-horizon GUI Agent Trajectory Synthesis

arXiv:2607.1804614.2MM
Predicted impact top 11% in LG · last 90 daysOriginality Incremental advance
AI Analysis

For GUI agent researchers, SEE provides a scalable method to generate high-coverage, long-horizon interaction data without costly human demonstrations.

SEE introduces a two-stage framework for synthesizing long-horizon GUI agent trajectories, achieving an average trajectory length of 14.8 steps without spurious loops, and fine-tuned agents show improved task success and generalization.

Graphical User Interface (GUI) agents powered by vision-language models hold promise for automating real-world mobile tasks. However, progress is limited by the lack of high-coverage, long-horizon interaction trajectories collected from element-rich and rapidly evolving apps. Existing pipelines often rely on costly human demonstrations or on-policy framework, which tends to over-sample common flows while missing rare transitions and complex multi-step procedures. To address this problem, we propose SEE, a two-stage data synthesis framework consisting of (i) an efficient exploration stage that builds an explicit UI transition graph over screens and elements, and (ii) a graph-based synthesis stage that composes diverse multi-step trajectories via planning and controlled sampling. This design yields reproducible and explainable data generation, while explicitly preventing spurious cycles and enabling long-horizon composition. Across multiple real-world apps, SEE produces trajectories with an average length of 14.8 steps while avoiding spurious loops, and agents fine-tuned on SEE achieve improved task success and generalization to unseen screens. We will publicly release our synthesis code and dataset.

Foundations

The foundational work for this paper's niche, ranked by how specifically the neighbourhood builds on it — not by global fame.

Your Notes