Unsupervised and interpretable scene discovery with Discrete-Attend-Infer-Repeat
This work addresses the problem of interpretable scene decomposition for researchers in unsupervised learning and computer vision, but it is incremental as it builds on the original AIR model with modifications for interpretability.
The paper tackles the problem of unsupervised and interpretable scene discovery by introducing Discrete-AIR, a recurrent auto-encoder with structured latent distributions, which efficiently identifies objects in images and provides direct interpretability of latent codes. The result shows that for Multi-MNIST and Multi-Sprites datasets, the model requires only one categorical latent variable, one attribute variable (for Multi-MNIST), and spatial attention variables for effective inference, with learned categorical distributions capturing object categories.
In this work we present Discrete Attend Infer Repeat (Discrete-AIR), a Recurrent Auto-Encoder with structured latent distributions containing discrete categorical distributions, continuous attribute distributions, and factorised spatial attention. While inspired by the original AIR model andretaining AIR model's capability in identifying objects in an image, Discrete-AIR provides direct interpretability of the latent codes. We show that for Multi-MNIST and a multiple-objects version of dSprites dataset, the Discrete-AIR model needs just one categorical latent variable, one attribute variable (for Multi-MNIST only), together with spatial attention variables, for efficient inference. We perform analysis to show that the learnt categorical distributions effectively capture the categories of objects in the scene for Multi-MNIST and for Multi-Sprites.