SLayR: Scene Layout Generation with Rectified Flow
This addresses a domain-specific problem for text-to-image generation where prompts lack constraints, enabling more varied and realistic scene layouts.
The paper tackles the problem of generating diverse and plausible scene layouts from ambiguous text prompts, where current text-to-image pipelines struggle, by introducing SLayR, a transformer-based model that surpasses existing baselines including LLMs and achieves state-of-the-art results while being at least 3x smaller in parameters.
We introduce SLayR, Scene Layout Generation with Rectified flow, a novel transformer-based model for text-to-layout generation which can then be paired with existing layout-to-image models to produce images. SLayR addresses a domain in which current text-to-image pipelines struggle: generating scene layouts that are of significant variety and plausibility, when the given prompt is ambiguous and does not provide constraints on the scene. SLayR surpasses existing baselines including LLMs in unconstrained generation, and can generate layouts from an open caption set. To accurately evaluate the layout generation, we introduce a new benchmark suite, including numerical metrics and a carefully designed repeatable human-evaluation procedure that assesses the plausibility and variety of generated images. We show that our method sets a new state of the art for achieving both at the same time, while being at least 3x times smaller in the number of parameters.