Ar2Can: An Architect and an Artist Leveraging a Canvas for Multi-Human Generation
This addresses a specific bottleneck in text-to-image generation for multi-human scenes, which is important for applications like digital art or content creation, though it appears incremental as it builds on existing diffusion models.
The paper tackles the problem of generating reliable multi-human scenes in text-to-image generation, where existing models often fail by duplicating faces or miscounting individuals. The result is Ar2Can, a two-stage framework that achieves substantial improvements in count accuracy and identity preservation on the MultiHuman-Testbench while maintaining high perceptual quality.
Despite recent advances in text-to-image generation, existing models consistently fail to produce reliable multi-human scenes, often duplicating faces, merging identities, or miscounting individuals. We present Ar2Can, a novel two-stage framework that disentangles spatial planning from identity rendering for multi-human generation. The Architect module predicts structured layouts, specifying where each person should appear. The Artist module then synthesizes photorealistic images, guided by a spatially-grounded face matching reward that combines Hungarian spatial alignment with ArcFace identity similarity. This approach ensures faces are rendered at correct locations and faithfully preserve reference identities. We develop two Architect variants, seamlessly integrated with our diffusion-based Artist model and optimized via Group Relative Policy Optimization (GRPO) using compositional rewards for count accuracy, image quality, and identity matching. Evaluated on the MultiHuman-Testbench, Ar2Can achieves substantial improvements in both count accuracy and identity preservation, while maintaining high perceptual quality. Notably, our method achieves these results using primarily synthetic data, without requiring real multi-human images.