Learning Goal-Oriented Visual Dialog via Tempered Policy Gradient
This work addresses suboptimal policies in goal-oriented visual dialogues for AI dialogue systems, representing an incremental improvement over existing methods.
The paper tackles the problem of goal-oriented visual dialogue agents focusing on simple utterances and suboptimal policies by proposing Tempered Policy Gradient (TPG) methods, achieving a 7% improvement with Seq2Seq and Memory Network extensions and an additional 5% improvement with TPG on the GuessWhat?! game.
Learning goal-oriented dialogues by means of deep reinforcement learning has recently become a popular research topic. However, commonly used policy-based dialogue agents often end up focusing on simple utterances and suboptimal policies. To mitigate this problem, we propose a class of novel temperature-based extensions for policy gradient methods, which are referred to as Tempered Policy Gradients (TPGs). On a recent AI-testbed, i.e., the GuessWhat?! game, we achieve significant improvements with two innovations. The first one is an extension of the state-of-the-art solutions with Seq2Seq and Memory Network structures that leads to an improvement of 7%. The second one is the application of our newly developed TPG methods, which improves the performance additionally by around 5% and, even more importantly, helps produce more convincing utterances.