Generating Multi-Image Synthetic Data for Text-to-Image Customization
This work addresses the challenge of multi-image supervision for text-to-image customization, offering a more efficient alternative to test-time optimization, though it is incremental in nature.
The paper tackles the problem of customizing text-to-image models by generating high-quality images from reference objects, proposing a method that improves upon existing encoder-based approaches on standard benchmarks.
Customization of text-to-image models enables users to insert new concepts or objects and generate them in unseen settings. Existing methods either rely on comparatively expensive test-time optimization or train encoders on single-image datasets without multi-image supervision, which can limit image quality. We propose a simple approach to address these challenges. We first leverage existing text-to-image models and 3D datasets to create a high-quality Synthetic Customization Dataset (SynCD) consisting of multiple images of the same object in different lighting, backgrounds, and poses. Using this dataset, we train an encoder-based model that incorporates fine-grained visual details from reference images via a shared attention mechanism. Finally, we propose an inference technique that normalizes text and image guidance vectors to mitigate overexposure issues in sampled images. Through extensive experiments, we show that our encoder-based model, trained on SynCD, and with the proposed inference algorithm, improves upon existing encoder-based methods on standard customization benchmarks.