Building Task-Oriented Visual Dialog Systems Through Alternative Optimization Between Dialog Policy and Language Generation
This work addresses the problem of designing reward functions for visual dialog systems, which is incremental as it builds on existing RL and seq2seq methods.
The paper tackles the challenge of balancing effective policy learning with natural dialog generation in task-oriented visual dialog systems by proposing a framework that alternates training between a reinforcement learning policy for image guessing and a supervised sequence-to-sequence model. It achieves state-of-the-art performance on the GuessWhich task in both task completion and dialog quality.
Reinforcement learning (RL) is an effective approach to learn an optimal dialog policy for task-oriented visual dialog systems. A common practice is to apply RL on a neural sequence-to-sequence (seq2seq) framework with the action space being the output vocabulary in the decoder. However, it is difficult to design a reward function that can achieve a balance between learning an effective policy and generating a natural dialog response. This paper proposes a novel framework that alternatively trains a RL policy for image guessing and a supervised seq2seq model to improve dialog generation quality. We evaluate our framework on the GuessWhich task and the framework achieves the state-of-the-art performance in both task completion and dialog quality.