NaRLE: Natural Language Models using Reinforcement Learning with Emotion Feedback
This work addresses the challenge of online adaptation for dialogue systems in specific domains like email handling, representing an incremental advancement in reinforcement learning applications for NLU.
The paper tackles the problem of improving natural language understanding in task-based conversational assistants for document-type conversations without human-labeled data, achieving up to 43% performance improvement in intent classification by using reinforcement learning with emotion feedback.
Current research in dialogue systems is focused on conversational assistants working on short conversations in either task-oriented or open domain settings. In this paper, we focus on improving task-based conversational assistants online, primarily those working on document-type conversations (e.g., emails) whose contents may or may not be completely related to the assistant's task. We propose "NARLE" a deep reinforcement learning (RL) framework for improving the natural language understanding (NLU) component of dialogue systems online without the need to collect human labels for customer data. The proposed solution associates user emotion with the assistant's action and uses that to improve NLU models using policy gradients. For two intent classification problems, we empirically show that using reinforcement learning to fine tune the pre-trained supervised learning models improves performance up to 43%. Furthermore, we demonstrate the robustness of the method to partial and noisy implicit feedback.