Interactive Reinforcement Learning for Object Grounding via Self-Talking
This work addresses the challenge of enabling AI agents to identify referred visual objects through natural language communication, which is incremental as it builds on existing interactive reinforcement learning approaches for visual grounding.
The paper tackles the problem of improving natural language conversation systems for visual grounding by introducing an interactive training method where both agents are reinforced by a common, self-updating reward function, achieving a significant improvement in task success rate on the GuessWhat?! dataset.
Humans are able to identify a referred visual object in a complex scene via a few rounds of natural language communications. Success communication requires both parties to engage and learn to adapt for each other. In this paper, we introduce an interactive training method to improve the natural language conversation system for a visual grounding task. During interactive training, both agents are reinforced by the guidance from a common reward function. The parametrized reward function also cooperatively updates itself via interactions, and contribute to accomplishing the task. We evaluate the method on GuessWhat?! visual grounding task, and significantly improve the task success rate. However, we observe language drifting problem during training and propose to use reward engineering to improve the interpretability for the generated conversations. Our result also indicates evaluating goal-ended visual conversation tasks require semantic relevant metrics beyond task success rate.