Generative Hierarchical Models for Parts, Objects, and Scenes
This work addresses the need for interpretable and transferable representations in downstream tasks, offering a novel approach to avoid routing issues in bottom-up methods.
The paper tackles the problem of modeling compositional hierarchies in natural scenes via unsupervised learning, proposing RICH, a deep latent variable model that learns interpretable hierarchical representations and generates imaginary scenes.
Compositional structures between parts and objects are inherent in natural scenes. Modeling such compositional hierarchies via unsupervised learning can bring various benefits such as interpretability and transferability, which are important in many downstream tasks. In this paper, we propose the first deep latent variable model, called RICH, for learning Representation of Interpretable Compositional Hierarchies. At the core of RICH is a latent scene graph representation that organizes the entities of a scene into a tree structure according to their compositional relationships. During inference, taking top-down approach, RICH is able to use higher-level representation to guide lower-level decomposition. This avoids the difficult problem of routing between parts and objects that is faced by bottom-up approaches. In experiments on images containing multiple objects with different part compositions, we demonstrate that RICH is able to learn the latent compositional hierarchy and generate imaginary scenes.