Interpretable Neural Computation for Real-World Compositional Visual Question Answering
This work addresses the challenge of making AI systems more transparent and reliable for real-world applications like visual question answering, though it is incremental in combining existing approaches.
The paper tackled the problem of combining interpretable compositional reasoning with high performance for real-world visual question answering, resulting in a framework that outperforms prior compositional methods and achieves competitive accuracy with monolithic networks on the GQA benchmark.
There are two main lines of research on visual question answering (VQA): compositional model with explicit multi-hop reasoning, and monolithic network with implicit reasoning in the latent feature space. The former excels in interpretability and compositionality but fails on real-world images, while the latter usually achieves better performance due to model flexibility and parameter efficiency. We aim to combine the two to build an interpretable framework for real-world compositional VQA. In our framework, images and questions are disentangled into scene graphs and programs, and a symbolic program executor runs on them with full transparency to select the attention regions, which are then iteratively passed to a visual-linguistic pre-trained encoder to predict answers. Experiments conducted on the GQA benchmark demonstrate that our framework outperforms the compositional prior arts and achieves competitive accuracy among monolithic ones. With respect to the validity, plausibility and distribution metrics, our framework surpasses others by a considerable margin.