Discrete Subgraph Sampling for Interpretable Graph based Visual Question Answering
This work addresses the problem of making AI models more transparent for users in visual question answering, though it is incremental as it builds on existing graph-based methods.
The paper tackled the performance trade-off between interpretability and answer accuracy in graph-based visual question answering by integrating discrete subset sampling methods to generate explanatory subgraphs intrinsically, achieving strong co-occurrences between answer and question tokens on the GQA dataset.
Explainable artificial intelligence (XAI) aims to make machine learning models more transparent. While many approaches focus on generating explanations post-hoc, interpretable approaches, which generate the explanations intrinsically alongside the predictions, are relatively rare. In this work, we integrate different discrete subset sampling methods into a graph-based visual question answering system to compare their effectiveness in generating interpretable explanatory subgraphs intrinsically. We evaluate the methods on the GQA dataset and show that the integrated methods effectively mitigate the performance trade-off between interpretability and answer accuracy, while also achieving strong co-occurrences between answer and question tokens. Furthermore, we conduct a human evaluation to assess the interpretability of the generated subgraphs using a comparative setting with the extended Bradley-Terry model, showing that the answer and question token co-occurrence metrics strongly correlate with human preferences. Our source code is publicly available.