Enhancing Visual Dialog State Tracking through Iterative Object-Entity Alignment in Multi-Round Conversations
This addresses the challenge of multi-round visual dialog for AI agents, though it appears incremental as it builds on existing methods by focusing on conversational information flows.
The paper tackles the problem of visual dialog state tracking by introducing a framework that leverages round-level dialog history to ground questions, achieving state-of-the-art performance on the VisDial v1.0 dataset in a generative setting.
Visual Dialog (VD) is a task where an agent answers a series of image-related questions based on a multi-round dialog history. However, previous VD methods often treat the entire dialog history as a simple text input, disregarding the inherent conversational information flows at the round level. In this paper, we introduce Multi-round Dialogue State Tracking model (MDST), a framework that addresses this limitation by leveraging the dialogue state learned from dialog history to answer questions. MDST captures each round of dialog history, constructing internal dialogue state representations defined as 2-tuples of vision-language representations. These representations effectively ground the current question, enabling the generation of accurate answers. Experimental results on the VisDial v1.0 dataset demonstrate that MDST achieves a new state-of-the-art performance in generative setting. Furthermore, through a series of human studies, we validate the effectiveness of MDST in generating long, consistent, and human-like answers while consistently answering a series of questions correctly.