Representative Feature Extraction During Diffusion Process for Sketch Extraction with One Example
This addresses the need for efficient and high-quality sketch generation in computer vision, though it appears incremental as it builds on existing diffusion models.
The paper tackles the problem of generating stylized sketches from images by introducing DiffSketch, a method that extracts representative features from a pretrained diffusion model and trains with only one manual drawing, achieving state-of-the-art performance in sketch extraction.
We introduce DiffSketch, a method for generating a variety of stylized sketches from images. Our approach focuses on selecting representative features from the rich semantics of deep features within a pretrained diffusion model. This novel sketch generation method can be trained with one manual drawing. Furthermore, efficient sketch extraction is ensured by distilling a trained generator into a streamlined extractor. We select denoising diffusion features through analysis and integrate these selected features with VAE features to produce sketches. Additionally, we propose a sampling scheme for training models using a conditional generative approach. Through a series of comparisons, we verify that distilled DiffSketch not only outperforms existing state-of-the-art sketch extraction methods but also surpasses diffusion-based stylization methods in the task of extracting sketches.