Asynchronous Tool Usage for Real-Time Agents
This addresses the limitation of reduced interactivity in AI systems for users requiring real-time multitasking, representing a novel approach rather than an incremental improvement.
The paper tackles the problem of synchronous, turn-based AI agents by introducing asynchronous agents capable of parallel processing and real-time tool-use, resulting in a framework for fluid, multitasking interactions.
While frontier large language models (LLMs) are capable tool-using agents, current AI systems still operate in a strict turn-based fashion, oblivious to passage of time. This synchronous design forces user queries and tool-use to occur sequentially, preventing the systems from multitasking and reducing interactivity. To address this limitation, we introduce asynchronous AI agents capable of parallel processing and real-time tool-use. Our key contribution is an event-driven finite-state machine architecture for agent execution and prompting, integrated with automatic speech recognition and text-to-speech. Drawing inspiration from the concepts originally developed for real-time operating systems, this work presents both a conceptual framework and practical tools for creating AI agents capable of fluid, multitasking interactions.