Lipstick ain't enough: Beyond Color Matching for In-the-Wild Makeup Transfer
This addresses the limitation of existing methods that only handle color manipulation, enabling more realistic and diverse makeup applications in computer vision.
The paper tackles the problem of makeup transfer beyond simple color matching to include patterns like stickers and blushes, achieving state-of-the-art performance on both light and extreme makeup styles.
Makeup transfer is the task of applying on a source face the makeup style from a reference image. Real-life makeups are diverse and wild, which cover not only color-changing but also patterns, such as stickers, blushes, and jewelries. However, existing works overlooked the latter components and confined makeup transfer to color manipulation, focusing only on light makeup styles. In this work, we propose a holistic makeup transfer framework that can handle all the mentioned makeup components. It consists of an improved color transfer branch and a novel pattern transfer branch to learn all makeup properties, including color, shape, texture, and location. To train and evaluate such a system, we also introduce new makeup datasets for real and synthetic extreme makeup. Experimental results show that our framework achieves the state of the art performance on both light and extreme makeup styles. Code is available at https://github.com/VinAIResearch/CPM.