Characterizing Software Aging in GPU-Based LLM Serving Systems
It addresses the lack of empirical aging studies for LLM serving systems, which are critical for reliability in production deployments.
This paper characterizes software aging in GPU-based LLM serving systems through a 216-hour campaign, revealing statistically significant memory aging across all deployments with leak rates strongly dependent on the serving runtime and configuration.
This paper proposes an empirical methodology to study software aging in GPU-based LLM serving systems. Traditional aging studies focus on CPU-centric software with relatively regular workloads; LLM serving is different, spanning a Python host and a CUDA device, handling requests whose cost varies by orders of magnitude, and relying on rapidly evolving software stacks. We run a 216-hour campaign across six co-located deployments under identical stress conditions, monitor host, device, and client metrics in parallel, and apply a statistical pipeline that accounts for autocorrelation and multiple testing. Our results reveal statistically significant memory aging in all deployments, with leak rates strongly dependent on the serving runtime and deployment configuration. Beyond these findings, we provide a reproducible framework that opens a research direction at the intersection of the software aging and rejuvenation and LLM serving communities.