Help Without Being Asked: A Deployed Proactive Agent System for On-Call Support with Continuous Self-Improvement
This addresses the issue of inefficient on-call support for cloud service platforms, offering a novel approach to reduce analyst workload, though it is incremental by building on existing reactive agent systems.
The paper tackles the problem of high workload on human support analysts in cloud service platforms by introducing Vigil, a proactive agent system that assists during on-call dialogues without explicit user invocation, resulting in deployment for over ten months with demonstrated effectiveness and practicality.
In large-scale cloud service platforms, thousands of customer tickets are generated daily and are typically handled through on-call dialogues. This high volume of on-call interactions imposes a substantial workload on human support analysts. Recent studies have explored reactive agents that leverage large language models as a first line of support to interact with customers directly and resolve issues. However, when issues remain unresolved and are escalated to human support, these agents are typically disengaged. As a result, they cannot assist with follow-up inquiries, track resolution progress, or learn from the cases they fail to address. In this paper, we introduce Vigil, a novel proactive agent system designed to operate throughout the entire on-call life-cycle. Unlike reactive agents, Vigil focuses on providing assistance during the phase in which human support is already involved. It integrates into the dialogue between the customer and the analyst, proactively offering assistance without explicit user invocation. Moreover, Vigil incorporates a continuous self-improvement mechanism that extracts knowledge from human-resolved cases to autonomously update its capabilities. Vigil has been deployed on Volcano Engine, ByteDance's cloud platform, for over ten months, and comprehensive evaluations based on this deployment demonstrate its effectiveness and practicality. The open source version of this work is publicly available at https://github.com/volcengine/veaiops.