Learning to Generate Multiple Objects from Dense and Occluded Layouts
For text-to-image generation, this work addresses the specific problem of generating multiple objects in dense and occluded layouts, which is a known bottleneck in current diffusion models.
Text-to-image diffusion models fail to generate correct object counts in dense, overlapping scenes due to instance ownership collapse. The proposed method, using layout-aware attention biases and an amodal-balanced loss, substantially improves count accuracy and prevents instance merging while preserving image quality.
Text-to-image diffusion models fail to generate correct object counts in dense scenes, where overlapping instances collapse into indistinguishable structures despite appearing visually plausible. We identify this as instance ownership collapse: tokens from overlapping objects interact freely through attention, while heavily occluded instances receive weak supervision due to their small visible areas. We address this through layout-aware attention biases that softly bias token interactions toward region-consistent grouping and suppress cross-instance leakage, paired with an amodal-balanced loss that amplifies gradients for occluded objects based on their occlusion level. To enable systematic evaluation, we introduce OverlapDepth-45K, a benchmark of densely overlapping scenes with amodal supervision. Our approach substantially improves count accuracy and prevents instance merging while preserving image quality. Project page: https://bachngoh.github.io/AIBL