DreamMakeup: Face Makeup Customization using Latent Diffusion Models
This addresses the need for better virtual makeup simulation tools in the growing global makeup market, offering a more stable and customizable alternative to GAN-based approaches.
The paper tackles the problem of virtual makeup customization by introducing DreamMakeup, a training-free diffusion model that enables precise real-image editing with various conditioning inputs like reference images, colors, and text, demonstrating notable improvements in customization, color-matching, and identity preservation over existing methods.
The exponential growth of the global makeup market has paralleled advancements in virtual makeup simulation technology. Despite the progress led by GANs, their application still encounters significant challenges, including training instability and limited customization capabilities. Addressing these challenges, we introduce DreamMakup - a novel training-free Diffusion model based Makeup Customization method, leveraging the inherent advantages of diffusion models for superior controllability and precise real-image editing. DreamMakeup employs early-stopped DDIM inversion to preserve the facial structure and identity while enabling extensive customization through various conditioning inputs such as reference images, specific RGB colors, and textual descriptions. Our model demonstrates notable improvements over existing GAN-based and recent diffusion-based frameworks - improved customization, color-matching capabilities, identity preservation and compatibility with textual descriptions or LLMs with affordable computational costs.