Face Sketch Synthesis via Semantic-Driven Generative Adversarial Network
This work addresses face sketch synthesis for applications in digital entertainment and law enforcement, representing an incremental improvement with novel components.
The paper tackled the challenge of generating accurate and realistic face sketches from photos under illumination variations and complex backgrounds by proposing a Semantic-Driven Generative Adversarial Network (SDGAN) with global structure-level style injection and local class-level knowledge re-weighting, achieving state-of-the-art performance on CUFS and CUFSF datasets.
Face sketch synthesis has made significant progress with the development of deep neural networks in these years. The delicate depiction of sketch portraits facilitates a wide range of applications like digital entertainment and law enforcement. However, accurate and realistic face sketch generation is still a challenging task due to the illumination variations and complex backgrounds in the real scenes. To tackle these challenges, we propose a novel Semantic-Driven Generative Adversarial Network (SDGAN) which embeds global structure-level style injection and local class-level knowledge re-weighting. Specifically, we conduct facial saliency detection on the input face photos to provide overall facial texture structure, which could be used as a global type of prior information. In addition, we exploit face parsing layouts as the semantic-level spatial prior to enforce globally structural style injection in the generator of SDGAN. Furthermore, to enhance the realistic effect of the details, we propose a novel Adaptive Re-weighting Loss (ARLoss) which dedicates to balance the contributions of different semantic classes. Experimentally, our extensive experiments on CUFS and CUFSF datasets show that our proposed algorithm achieves state-of-the-art performance.