LayoutDETR: Detection Transformer Is a Good Multimodal Layout Designer
This addresses the need for scalable, automated layout design for visual communication, reducing the time and skill required for handcrafting, though it is incremental as it builds on existing generative and detection methods.
The paper tackles the problem of automating graphic layout design by generating layouts that comply with multimodal constraints from background images and foreground content, achieving new state-of-the-art performance on public benchmarks and a new ad banner dataset, with user studies showing significant preference over baselines.
Graphic layout designs play an essential role in visual communication. Yet handcrafting layout designs is skill-demanding, time-consuming, and non-scalable to batch production. Generative models emerge to make design automation scalable but it remains non-trivial to produce designs that comply with designers' multimodal desires, i.e., constrained by background images and driven by foreground content. We propose LayoutDETR that inherits the high quality and realism from generative modeling, while reformulating content-aware requirements as a detection problem: we learn to detect in a background image the reasonable locations, scales, and spatial relations for multimodal foreground elements in a layout. Our solution sets a new state-of-the-art performance for layout generation on public benchmarks and on our newly-curated ad banner dataset. We integrate our solution into a graphical system that facilitates user studies, and show that users prefer our designs over baselines by significant margins. Code, models, dataset, and demos are available at https://github.com/salesforce/LayoutDETR.