Key-point Guided Deformable Image Manipulation Using Diffusion Model
It addresses the problem of fine-grained control in image generation for applications such as medical imaging and animation, though it is incremental by building on existing diffusion models.
The paper tackles precise image manipulation by controlling object key-points using a two-stage diffusion model with optical flow, achieving more realistic and consistent image generation across tasks like facial images and human pose synthesis.
In this paper, we introduce a Key-point-guided Diffusion probabilistic Model (KDM) that gains precise control over images by manipulating the object's key-point. We propose a two-stage generative model incorporating an optical flow map as an intermediate output. By doing so, a dense pixel-wise understanding of the semantic relation between the image and sparse key point is configured, leading to more realistic image generation. Additionally, the integration of optical flow helps regulate the inter-frame variance of sequential images, demonstrating an authentic sequential image generation. The KDM is evaluated with diverse key-point conditioned image synthesis tasks, including facial image generation, human pose synthesis, and echocardiography video prediction, demonstrating the KDM is proving consistency enhanced and photo-realistic images compared with state-of-the-art models.