Enhancing Customer Service Chatbots with Context-Aware NLU through Selective Attention and Multi-task Learning
This work addresses the problem of low-accuracy intent classification for customer service chatbots, particularly for ambiguous queries, which is incremental as it builds on existing methods by adding context and multi-task learning.
The paper tackles the problem of ambiguous queries in customer service chatbots by introducing a context-aware NLU model that uses selective attention and multi-task learning to incorporate customer order status, resulting in a 4.8% improvement in top-2 accuracy over a baseline and 3.5% over state-of-the-art models, with deployment at Walmart leading to significant cost savings.
Customer service chatbots are conversational systems aimed at addressing customer queries, often by directing them to automated workflows. A crucial aspect of this process is the classification of the customer's intent. Presently, most intent classification models for customer care utilise only customer query for intent prediction. This may result in low-accuracy models, which cannot handle ambiguous queries. An ambiguous query like "I didn't receive my package" could indicate a delayed order, or an order that was delivered but the customer failed to receive it. Resolution of each of these scenarios requires the execution of very different sequence of steps. Utilizing additional information, such as the customer's order delivery status, in the right manner can help identify the intent for such ambiguous queries. In this paper, we have introduced a context-aware NLU model that incorporates both, the customer query and contextual information from the customer's order status for predicting customer intent. A novel selective attention module is used to extract relevant context features. We have also proposed a multi-task learning paradigm for the effective utilization of different label types available in our training data. Our suggested method, Multi-Task Learning Contextual NLU with Selective Attention Weighted Context (MTL-CNLU-SAWC), yields a 4.8% increase in top 2 accuracy score over the baseline model which only uses user queries, and a 3.5% improvement over existing state-of-the-art models that combine query and context. We have deployed our model to production for Walmart's customer care domain. Accurate intent prediction through MTL-CNLU-SAWC helps to better direct customers to automated workflows, thereby significantly reducing escalations to human agents, leading to almost a million dollars in yearly savings for the company.