Chimera: Compositional Image Generation using Part-based Concepting
This addresses the need for explicit control in personalized image generation for users, though it is incremental as it builds on existing generative models.
The paper tackles the problem of generating personalized images by combining specified parts from multiple source images without requiring user-provided masks or annotations, achieving a 14% improvement in part alignment and compositional accuracy and a 21% improvement in visual quality over baselines.
Personalized image generative models are highly proficient at synthesizing images from text or a single image, yet they lack explicit control for composing objects from specific parts of multiple source images without user specified masks or annotations. To address this, we introduce Chimera, a personalized image generation model that generates novel objects by combining specified parts from different source images according to textual instructions. To train our model, we first construct a dataset from a taxonomy built on 464 unique (part, subject) pairs, which we term semantic atoms. From this, we generate 37k prompts and synthesize the corresponding images with a high-fidelity text-to-image model. We train a custom diffusion prior model with part-conditional guidance, which steers the image-conditioning features to enforce both semantic identity and spatial layout. We also introduce an objective metric PartEval to assess the fidelity and compositional accuracy of generation pipelines. Human evaluations and our proposed metric show that Chimera outperforms other baselines by 14% in part alignment and compositional accuracy and 21% in visual quality.