Sketch-to-Art: Synthesizing Stylized Art Images From Sketches
This addresses the challenge of automating artistic image synthesis for creators, but it is incremental as it builds on existing GAN frameworks.
The paper tackles the problem of generating detailed art-stylized images from sketches without semantic tags, using a reference image for style, and achieves a significant qualitative and quantitative improvement over state-of-the-art baselines.
We propose a new approach for synthesizing fully detailed art-stylized images from sketches. Given a sketch, with no semantic tagging, and a reference image of a specific style, the model can synthesize meaningful details with colors and textures. The model consists of three modules designed explicitly for better artistic style capturing and generation. Based on a GAN framework, a dual-masked mechanism is introduced to enforce the content constraints (from the sketch), and a feature-map transformation technique is developed to strengthen the style consistency (to the reference image). Finally, an inverse procedure of instance-normalization is proposed to disentangle the style and content information, therefore yields better synthesis performance. Experiments demonstrate a significant qualitative and quantitative boost over baselines based on previous state-of-the-art techniques, adopted for the proposed process.