Mixture of Diffusers for scene composition and high resolution image generation
This addresses a limitation in text-to-image generation for users needing precise scene composition, though it appears incremental as it builds over existing diffusion models.
The paper tackles the problem of generating specific image compositions with diffusion models by introducing Mixture of Diffusers, which harmonizes multiple diffusion processes on different canvas regions to control object and style locations, resulting in larger images with detailed composition control.
Diffusion methods have been proven to be very effective to generate images while conditioning on a text prompt. However, and although the quality of the generated images is unprecedented, these methods seem to struggle when trying to generate specific image compositions. In this paper we present Mixture of Diffusers, an algorithm that builds over existing diffusion models to provide a more detailed control over composition. By harmonizing several diffusion processes acting on different regions of a canvas, it allows generating larger images, where the location of each object and style is controlled by a separate diffusion process.