Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning
This work addresses the problem of enabling more natural and dynamic human-machine interactions for users of conversational AI systems, though it is incremental as it builds on existing language models and RL techniques.
The paper tackled the challenge of building automated agents for rich open-ended conversations by developing a real-time dialogue system using reinforcement learning, which substantially exceeded a strong supervised baseline in a live experiment with real users on Google Assistant.
Despite recent advances in natural language understanding and generation, and decades of research on the development of conversational bots, building automated agents that can carry on rich open-ended conversations with humans "in the wild" remains a formidable challenge. In this work we develop a real-time, open-ended dialogue system that uses reinforcement learning (RL) to power a bot's conversational skill at scale. Our work pairs the succinct embedding of the conversation state generated using SOTA (supervised) language models with RL techniques that are particularly suited to a dynamic action space that changes as the conversation progresses. Trained using crowd-sourced data, our novel system is able to substantially exceeds the (strong) baseline supervised model with respect to several metrics of interest in a live experiment with real users of the Google Assistant.