Intelligent Painter: Picture Composition With Resampling Diffusion Model
This addresses the need for controllable image generation for users wanting to create scenes with precise object placement, representing an incremental advance over existing diffusion models.
The paper tackles the problem of generating images from user-specified objects at specific locations, unlike standard inpainting, by proposing a resampling strategy for DDPM to compose harmonized pictures. Results show it produces less blurry, higher perceptual quality images compared to state-of-the-art methods, with quantitative improvements in image quality assessment.
Have you ever thought that you can be an intelligent painter? This means that you can paint a picture with a few expected objects in mind, or with a desirable scene. This is different from normal inpainting approaches for which the location of specific objects cannot be determined. In this paper, we present an intelligent painter that generate a person's imaginary scene in one go, given explicit hints. We propose a resampling strategy for Denoising Diffusion Probabilistic Model (DDPM) to intelligently compose unconditional harmonized pictures according to the input subjects at specific locations. By exploiting the diffusion property, we resample efficiently to produce realistic pictures. Experimental results show that our resampling method favors the semantic meaning of the generated output efficiently and generates less blurry output. Quantitative analysis of image quality assessment shows that our method produces higher perceptual quality images compared with the state-of-the-art methods.