History for Visual Dialog: Do we really need it?
This work addresses visual dialog systems for AI applications, but it is incremental as it builds on existing datasets and methods while exposing limitations.
The paper tackles the problem of visual dialog by showing that models explicitly encoding dialog history outperform those that don't, achieving 72% NDCG on the validation set, but also reveals dataset and metric issues, proposing a challenging subset with a benchmark of 63% NDCG.
Visual Dialog involves "understanding" the dialog history (what has been discussed previously) and the current question (what is asked), in addition to grounding information in the image, to generate the correct response. In this paper, we show that co-attention models which explicitly encode dialog history outperform models that don't, achieving state-of-the-art performance (72 % NDCG on val set). However, we also expose shortcomings of the crowd-sourcing dataset collection procedure by showing that history is indeed only required for a small amount of the data and that the current evaluation metric encourages generic replies. To that end, we propose a challenging subset (VisDialConv) of the VisDial val set and provide a benchmark of 63% NDCG.