A Neural-Symbolic Architecture for Inverse Graphics Improved by Lifelong Meta-Learning
This work addresses vision tasks by combining neural and symbolic approaches, but it appears incremental as it builds on existing inverse graphics and capsule network ideas.
The paper tackles the problem of vision as inverse graphics by introducing neural-symbolic capsules that de-render scenes into semantic information and render them back, with lifelong meta-learning used to improve detection capabilities through few-shot learning for new objects.
We follow the idea of formulating vision as inverse graphics and propose a new type of element for this task, a neural-symbolic capsule. It is capable of de-rendering a scene into semantic information feed-forward, as well as rendering it feed-backward. An initial set of capsules for graphical primitives is obtained from a generative grammar and connected into a full capsule network. Lifelong meta-learning continuously improves this network's detection capabilities by adding capsules for new and more complex objects it detects in a scene using few-shot learning. Preliminary results demonstrate the potential of our novel approach.