Customer Sentiment Analysis using Weak Supervision for Customer-Agent Chat
This work addresses sentiment analysis for customer support chats, an under-explored area due to data scarcity, but it is incremental as it applies existing weak supervision methods to a new domain.
The paper tackles sentiment analysis in customer-agent chat data using weak supervision, achieving a fairly accurate classifier by combining weak sentiment classifiers and domain-specific lexicon-based rules, and shows it outperforms an off-the-shelf Google Cloud NLP API for domain-specific cases.
Prior work on sentiment analysis using weak supervision primarily focuses on different reviews such as movies (IMDB), restaurants (Yelp), products (Amazon).~One under-explored field in this regard is customer chat data for a customer-agent chat in customer support due to the lack of availability of free public data. Here, we perform sentiment analysis on customer chat using weak supervision on our in-house dataset. We fine-tune the pre-trained language model (LM) RoBERTa as a sentiment classifier using weak supervision. Our contribution is as follows:1) We show that by using weak sentiment classifiers along with domain-specific lexicon-based rules as Labeling Functions (LF), we can train a fairly accurate customer chat sentiment classifier using weak supervision. 2) We compare the performance of our custom-trained model with off-the-shelf google cloud NLP API for sentiment analysis. We show that by injecting domain-specific knowledge using LFs, even with weak supervision, we can train a model to handle some domain-specific use cases better than off-the-shelf google cloud NLP API. 3) We also present an analysis of how customer sentiment in a chat relates to problem resolution.