MaskAttn-SDXL: Controllable Region-Level Text-To-Image Generation
This addresses the problem of generating accurate multi-object scenes in text-to-image models, offering a practical extension for spatial control with negligible overhead.
The paper tackles compositional failures in text-to-image diffusion models by proposing MaskAttn-SDXL, a region-level gating mechanism that sparsifies cross-attention logits to reduce cross-token interference, resulting in improved spatial compliance and attribute binding for multi-object prompts.
Text-to-image diffusion models achieve impressive realism but often suffer from compositional failures on prompts with multiple objects, attributes, and spatial relations, resulting in cross-token interference where entities entangle, attributes mix across objects, and spatial cues are violated. To address these failures, we propose MaskAttn-SDXL,a region-level gating mechanism applied to the cross-attention logits of Stable Diffusion XL(SDXL)'s UNet. MaskAttn-SDXL learns a binary mask per layer, injecting it into each cross-attention logit map before softmax to sparsify token-to-latent interactions so that only semantically relevant connections remain active. The method requires no positional encodings, auxiliary tokens, or external region masks, and preserves the original inference path with negligible overhead. In practice, our model improves spatial compliance and attribute binding in multi-object prompts while preserving overall image quality and diversity. These findings demonstrate that logit-level maksed cross-attention is an data-efficient primitve for enforcing compositional control, and our method thus serves as a practical extension for spatial control in text-to-image generation.