Bridging Unpaired Facial Photos And Sketches By Line-drawings
This addresses the challenge of creating realistic sketches from facial photos without paired data, which is useful for applications like digital art and forensics, but it is incremental as it builds on neural style transfer and unpaired translation methods.
The paper tackles the problem of face sketch synthesis from unpaired photos and sketches by introducing a line-drawing domain as an intermediate step, resulting in a method that significantly outperforms existing unpaired image-to-image translation techniques and generates multi-style sketches.
In this paper, we propose a novel method to learn face sketch synthesis models by using unpaired data. Our main idea is bridging the photo domain $\mathcal{X}$ and the sketch domain $Y$ by using the line-drawing domain $\mathcal{Z}$. Specially, we map both photos and sketches to line-drawings by using a neural style transfer method, i.e. $F: \mathcal{X}/\mathcal{Y} \mapsto \mathcal{Z}$. Consequently, we obtain \textit{pseudo paired data} $(\mathcal{Z}, \mathcal{Y})$, and can learn the mapping $G:\mathcal{Z} \mapsto \mathcal{Y}$ in a supervised learning manner. In the inference stage, given a facial photo, we can first transfer it to a line-drawing and then to a sketch by $G \circ F$. Additionally, we propose a novel stroke loss for generating different types of strokes. Our method, termed sRender, accords well with human artists' rendering process. Experimental results demonstrate that sRender can generate multi-style sketches, and significantly outperforms existing unpaired image-to-image translation methods.