EmbeWebAgent: Embedding Web Agents into Any Customized UI
This addresses the need for more robust and expressive web agents in enterprise settings where frontend and backend control is available, though it appears incremental as it builds on existing agent concepts with specific UI integration.
The paper tackles the problem of web agents being limited by human interface-level observations by introducing EmbeWebAgent, a framework that embeds agents directly into existing UIs using frontend hooks and a backend workflow, resulting in minimal retrofitting effort and robust multi-step behaviors in live UI settings.
Most web agents operate at the human interface level, observing screenshots or raw DOM trees without application-level access, which limits robustness and action expressiveness. In enterprise settings, however, explicit control of both the frontend and backend is available. We present EmbeWebAgent, a framework for embedding agents directly into existing UIs using lightweight frontend hooks (curated ARIA and URL-based observations, and a per-page function registry exposed via a WebSocket) and a reusable backend workflow that performs reasoning and takes actions. EmbeWebAgent is stack-agnostic (e.g., React or Angular), supports mixed-granularity actions ranging from GUI primitives to higher-level composites, and orchestrates navigation, manipulation, and domain-specific analytics via MCP tools. Our demo shows minimal retrofitting effort and robust multi-step behaviors grounded in a live UI setting. Live Demo: https://youtu.be/Cy06Ljee1JQ