GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generation
This work addresses the need for efficient and flexible data generation for object detection tasks, particularly in domains like self-driving, but it is incremental as it builds on existing text-to-image diffusion models.
The paper tackles the problem of generating high-quality object detection data by proposing GeoDiffusion, a framework that translates geometric conditions like bounding boxes and camera views into text prompts for pre-trained diffusion models, achieving better performance than previous layout-to-image methods while being 4x faster in training.
Diffusion models have attracted significant attention due to the remarkable ability to create content and generate data for tasks like image classification. However, the usage of diffusion models to generate the high-quality object detection data remains an underexplored area, where not only image-level perceptual quality but also geometric conditions such as bounding boxes and camera views are essential. Previous studies have utilized either copy-paste synthesis or layout-to-image (L2I) generation with specifically designed modules to encode the semantic layouts. In this paper, we propose the GeoDiffusion, a simple framework that can flexibly translate various geometric conditions into text prompts and empower pre-trained text-to-image (T2I) diffusion models for high-quality detection data generation. Unlike previous L2I methods, our GeoDiffusion is able to encode not only the bounding boxes but also extra geometric conditions such as camera views in self-driving scenes. Extensive experiments demonstrate GeoDiffusion outperforms previous L2I methods while maintaining 4x training time faster. To the best of our knowledge, this is the first work to adopt diffusion models for layout-to-image generation with geometric conditions and demonstrate that L2I-generated images can be beneficial for improving the performance of object detectors.