Interactive Speculative Planning: Enhance Agent Efficiency through Co-design of System and User Interface
This work aims to improve the efficiency of LLM-based agents for users by reducing planning latency, which is an incremental improvement to existing agent systems.
This paper addresses the substantial planning latency in LLM-based agents, which stems from LLM efficiency limitations and the complexity of generating intermediate thoughts. The authors propose Interactive Speculative Planning, a human-centered method that co-designs the agent system and user interface to manage user interactions and interruptions, leveraging human input to expedite the planning process.
Agents, as user-centric tools, are increasingly deployed for human task delegation, assisting with a broad spectrum of requests by generating thoughts, engaging with user proxies, and producing action plans. However, agents based on large language models (LLMs) often face substantial planning latency due to two primary factors: the efficiency limitations of the underlying LLMs due to their large size and high demand, and the structural complexity of the agents due to the extensive generation of intermediate thoughts to produce the final output. Given that inefficiency in service provision can undermine the value of automation for users, this paper presents a human-centered efficient agent planning method -- Interactive Speculative Planning -- aiming at enhancing the efficiency of agent planning through both system design and human-AI interaction. Our approach advocates for the co-design of the agent system and user interface, underscoring the importance of an agent system that can fluidly manage user interactions and interruptions. By integrating human interruptions as a fundamental component of the system, we not only make it more user-centric but also expedite the entire process by leveraging human-in-the-loop interactions to provide accurate intermediate steps. Code and data will be released.