Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective
This addresses performance bottlenecks in scalable LLM-based agents for AI systems, representing an incremental improvement by applying data systems principles to a known problem.
The paper tackled inefficiencies in serving agentic workflows composed of interdependent LLM calls by introducing Helium, a workflow-aware framework that models workloads as query plans and uses caching and scheduling to maximize reuse, achieving up to 1.56x speedup over state-of-the-art systems.
Agentic workflows are composed of sequences of interdependent Large Language Model (LLM) calls, and they have become a dominant workload in modern AI systems. These workflows exhibit extensive redundancy from overlapping prompts and intermediate results due to speculative and parallel exploration. Existing LLM serving systems, such as vLLM, focus on optimizing individual inference calls and overlook cross-call dependencies, leading to significant inefficiencies. This paper rethinks LLM and agent serving from a data systems perspective and introduces Helium, a workflow-aware serving framework that models agentic workloads as query plans and treats LLM invocations as first-class operators. Helium integrates proactive caching and cache-aware scheduling to maximize reuse across prompts, KV states, and workflows. Through these techniques, Helium bridges classic query optimization principles with LLM serving, achieving up to 1.56x speedup over state-of-the-art agent serving systems on various workloads. Our results demonstrate that end-to-end optimization across workflows is essential for scalable and efficient LLM-based agents.