RAG based Question-Answering for Contextual Response Prediction System
This work addresses the challenge of reducing hallucinations and improving response quality for customer service agents in retail contact centers, though it is incremental as it applies known RAG techniques to a specific domain.
The paper tackled the problem of providing precise responses to customer queries in industry settings by developing a Retrieval Augmented Generation (RAG) framework with LLMs, which outperformed existing BERT-based algorithms in accuracy and relevance through automated and human evaluations.
Large Language Models (LLMs) have shown versatility in various Natural Language Processing (NLP) tasks, including their potential as effective question-answering systems. However, to provide precise and relevant information in response to specific customer queries in industry settings, LLMs require access to a comprehensive knowledge base to avoid hallucinations. Retrieval Augmented Generation (RAG) emerges as a promising technique to address this challenge. Yet, developing an accurate question-answering framework for real-world applications using RAG entails several challenges: 1) data availability issues, 2) evaluating the quality of generated content, and 3) the costly nature of human evaluation. In this paper, we introduce an end-to-end framework that employs LLMs with RAG capabilities for industry use cases. Given a customer query, the proposed system retrieves relevant knowledge documents and leverages them, along with previous chat history, to generate response suggestions for customer service agents in the contact centers of a major retail company. Through comprehensive automated and human evaluations, we show that this solution outperforms the current BERT-based algorithms in accuracy and relevance. Our findings suggest that RAG-based LLMs can be an excellent support to human customer service representatives by lightening their workload.