LayoutTransformer: Layout Generation and Completion with Self-attention
It addresses layout generation for various design and analysis tasks, but the approach is incremental as it adapts self-attention to a known bottleneck in layout modeling.
The paper tackles the problem of scene layout generation across diverse domains like images, documents, and 3D objects by proposing LayoutTransformer, a framework using self-attention to learn relationships between layout elements and generate novel layouts, achieving competitive performance on datasets such as COCO, PubLayNet, RICO, and Part-Net.
We address the problem of scene layout generation for diverse domains such as images, mobile applications, documents, and 3D objects. Most complex scenes, natural or human-designed, can be expressed as a meaningful arrangement of simpler compositional graphical primitives. Generating a new layout or extending an existing layout requires understanding the relationships between these primitives. To do this, we propose LayoutTransformer, a novel framework that leverages self-attention to learn contextual relationships between layout elements and generate novel layouts in a given domain. Our framework allows us to generate a new layout either from an empty set or from an initial seed set of primitives, and can easily scale to support an arbitrary of primitives per layout. Furthermore, our analyses show that the model is able to automatically capture the semantic properties of the primitives. We propose simple improvements in both representation of layout primitives, as well as training methods to demonstrate competitive performance in very diverse data domains such as object bounding boxes in natural images(COCO bounding box), documents (PubLayNet), mobile applications (RICO dataset) as well as 3D shapes (Part-Net). Code and other materials will be made available at https://kampta.github.io/layout.