Object-based reasoning in VQA
This work addresses the AI-complete challenge of multi-modal reasoning in VQA, but it is incremental as it builds on existing methods for object detection and reasoning.
The paper tackled the problem of Visual Question Answering (VQA) by combining object detection and reasoning modules to improve performance on complex tasks, achieving a few percent accuracy improvement on the CLEVR dataset's counting task.
Visual Question Answering (VQA) is a novel problem domain where multi-modal inputs must be processed in order to solve the task given in the form of a natural language. As the solutions inherently require to combine visual and natural language processing with abstract reasoning, the problem is considered as AI-complete. Recent advances indicate that using high-level, abstract facts extracted from the inputs might facilitate reasoning. Following that direction we decided to develop a solution combining state-of-the-art object detection and reasoning modules. The results, achieved on the well-balanced CLEVR dataset, confirm the promises and show significant, few percent improvements of accuracy on the complex "counting" task.