Incremental Summarization for Customer Support via Progressive Note-Taking and Agent Feedback
This addresses productivity issues for customer support agents, though it is incremental as it builds on existing summarization and feedback techniques.
The paper tackles the problem of customer support agents' context-switching effort by introducing an incremental summarization system that generates concise bullet notes during conversations, achieving a 3% reduction in case handling time (up to 9% in complex cases) and high agent satisfaction.
We introduce an incremental summarization system for customer support agents that intelligently determines when to generate concise bullet notes during conversations, reducing agents' context-switching effort and redundant review. Our approach combines a fine-tuned Mixtral-8x7B model for continuous note generation with a DeBERTa-based classifier to filter trivial content. Agent edits refine the online notes generation and regularly inform offline model retraining, closing the agent edits feedback loop. Deployed in production, our system achieved a 3% reduction in case handling time compared to bulk summarization (with reductions of up to 9% in highly complex cases), alongside high agent satisfaction ratings from surveys. These results demonstrate that incremental summarization with continuous feedback effectively enhances summary quality and agent productivity at scale.