End-to-end optimization of goal-driven and visually grounded dialogue systems
This addresses the challenge of moving beyond simplistic supervised learning approaches for dialogue systems to handle planning and grounding, which is incremental in applying reinforcement learning to this domain.
The paper tackled the problem of generating natural dialogues and discovering specific objects in complex pictures by introducing a Deep Reinforcement Learning method for visually grounded task-oriented dialogues, achieving encouraging results on a dataset of 120k dialogues.
End-to-end design of dialogue systems has recently become a popular research topic thanks to powerful tools such as encoder-decoder architectures for sequence-to-sequence learning. Yet, most current approaches cast human-machine dialogue management as a supervised learning problem, aiming at predicting the next utterance of a participant given the full history of the dialogue. This vision is too simplistic to render the intrinsic planning problem inherent to dialogue as well as its grounded nature, making the context of a dialogue larger than the sole history. This is why only chit-chat and question answering tasks have been addressed so far using end-to-end architectures. In this paper, we introduce a Deep Reinforcement Learning method to optimize visually grounded task-oriented dialogues, based on the policy gradient algorithm. This approach is tested on a dataset of 120k dialogues collected through Mechanical Turk and provides encouraging results at solving both the problem of generating natural dialogues and the task of discovering a specific object in a complex picture.