Do Explanations make VQA Models more Predictable to a Human?
This challenges the effectiveness of current explanation methods for improving human predictability in VQA, which is crucial for transparency in AI systems.
The study investigated whether existing explanations make Visual Question Answering (VQA) models more predictable to humans, finding that they do not, but human-in-the-loop black-box approaches do.
A rich line of research attempts to make deep neural networks more transparent by generating human-interpretable 'explanations' of their decision process, especially for interactive tasks like Visual Question Answering (VQA). In this work, we analyze if existing explanations indeed make a VQA model -- its responses as well as failures -- more predictable to a human. Surprisingly, we find that they do not. On the other hand, we find that human-in-the-loop approaches that treat the model as a black-box do.