COTA: Improving the Speed and Accuracy of Customer Support through Ranking and Deep Networks
This addresses the need for faster and more accurate customer support for companies and end users, though it is incremental as it builds on existing machine learning and NLP techniques.
The paper tackles the problem of automating customer support ticket classification and answer selection to improve speed and reliability, proposing COTA v1 and v2 systems, with COTA v2 outperforming v1 and an A/B test showing a 10% reduction in issue resolution time without reducing customer satisfaction.
For a company looking to provide delightful user experiences, it is of paramount importance to take care of any customer issues. This paper proposes COTA, a system to improve speed and reliability of customer support for end users through automated ticket classification and answers selection for support representatives. Two machine learning and natural language processing techniques are demonstrated: one relying on feature engineering (COTA v1) and the other exploiting raw signals through deep learning architectures (COTA v2). COTA v1 employs a new approach that converts the multi-classification task into a ranking problem, demonstrating significantly better performance in the case of thousands of classes. For COTA v2, we propose an Encoder-Combiner-Decoder, a novel deep learning architecture that allows for heterogeneous input and output feature types and injection of prior knowledge through network architecture choices. This paper compares these models and their variants on the task of ticket classification and answer selection, showing model COTA v2 outperforms COTA v1, and analyzes their inner workings and shortcomings. Finally, an A/B test is conducted in a production setting validating the real-world impact of COTA in reducing issue resolution time by 10 percent without reducing customer satisfaction.