LayoutVAE: Stochastic Scene Layout Generation From a Label Set
This addresses the need for plausible visual variations in scene generation for researchers, though it appears incremental as it builds on existing VAE methods.
The paper tackles the problem of generating varied scene layouts from a set of labels, proposing LayoutVAE, a variational autoencoder framework that produces stochastic layouts and detects unusual ones, with effectiveness verified on MNIST-Layouts and COCO 2017 Panoptic datasets.
Recently there is an increasing interest in scene generation within the research community. However, models used for generating scene layouts from textual description largely ignore plausible visual variations within the structure dictated by the text. We propose LayoutVAE, a variational autoencoder based framework for generating stochastic scene layouts. LayoutVAE is a versatile modeling framework that allows for generating full image layouts given a label set, or per label layouts for an existing image given a new label. In addition, it is also capable of detecting unusual layouts, potentially providing a way to evaluate layout generation problem. Extensive experiments on MNIST-Layouts and challenging COCO 2017 Panoptic dataset verifies the effectiveness of our proposed framework.