Unified Agent: Managing Interactions across Devices
This work provides a more robust and effective agent architecture for managing user interactions across multiple devices and over time, which is a growing problem for AI agent developers.
The paper addresses the challenge of AI agents operating across multiple devices and over time, where observations are fragmented. They propose a stateful agent design, Unified Agent, which effectively manages interaction evidence, facts, and requests in a compact form. This system significantly outperforms four adapted published designs in a new cross-device, cross-time benchmark, and its advantage holds across different MLLM families, capabilities, and reasoning efforts.
As capabilities rapidly increase, AI agents can move from running inside one app to acting across a user's devices over time. Yet existing agent systems still fall short in this scenario. This is because observations are scattered across devices and moments, but mainstream systems are not designed around this fact: a single agent that treats devices as tools lacks effective state management for all devices across time, and multi-agent systems coordinate across agents but do not maintain the compact carried state a cross-device, cross-time request needs. We argue that the agent should maintain an effectively designed state that organizes engagement evidence, stated facts, and the standing request in a compact, action-ready form for deciding its action given the current observation. To compare state designs, we construct a benchmark of user-agent interaction across devices and time. We instantiate this principle in Unified Agent, a stateful agent that carries interaction evidence across devices and moments and uses it with the current observation to act. In the default setting, it significantly outperforms our adaptations of four published designs. Across changes in multimodal large language model (MLLM) family, capability, and reasoning effort, it remains ahead of all compared systems, demonstrating that the state-design advantage is robust across MLLM settings. Our code and data will be publicly available on GitHub.