Hyper-dimensional computing for a visual question-answering system that is trainable end-to-end
This work addresses the challenge of integrating logical reasoning with neural networks for visual question answering, though it appears incremental in its approach.
The authors tackled visual question answering by proposing a system that uses hyper-dimensional computing to represent a differentiable knowledge base, enabling end-to-end training.
In this work we propose a system for visual question answering. Our architecture is composed of two parts, the first part creates the logical knowledge base given the image. The second part evaluates questions against the knowledge base. Differently from previous work, the knowledge base is represented using hyper-dimensional computing. This choice has the advantage that all the operations in the system, namely creating the knowledge base and evaluating the questions against it, are differentiable, thereby making the system easily trainable in an end-to-end fashion.