OverLayBench: A Benchmark for Layout-to-Image Generation with Dense Overlaps
This addresses a specific challenge in layout-to-image generation for computer vision applications, but it is incremental as it focuses on benchmarking and a fine-tuned model rather than a major breakthrough.
The paper tackles the problem of layout-to-image generation struggling with overlapping bounding boxes by introducing OverLayBench, a benchmark with balanced complexity, and OverLayScore, a metric quantifying overlap complexity, showing that existing benchmarks are biased toward simpler cases.
Despite steady progress in layout-to-image generation, current methods still struggle with layouts containing significant overlap between bounding boxes. We identify two primary challenges: (1) large overlapping regions and (2) overlapping instances with minimal semantic distinction. Through both qualitative examples and quantitative analysis, we demonstrate how these factors degrade generation quality. To systematically assess this issue, we introduce OverLayScore, a novel metric that quantifies the complexity of overlapping bounding boxes. Our analysis reveals that existing benchmarks are biased toward simpler cases with low OverLayScore values, limiting their effectiveness in evaluating model performance under more challenging conditions. To bridge this gap, we present OverLayBench, a new benchmark featuring high-quality annotations and a balanced distribution across different levels of OverLayScore. As an initial step toward improving performance on complex overlaps, we also propose CreatiLayout-AM, a model fine-tuned on a curated amodal mask dataset. Together, our contributions lay the groundwork for more robust layout-to-image generation under realistic and challenging scenarios. Project link: https://mlpc-ucsd.github.io/OverLayBench.