SketchDesc: Learning Local Sketch Descriptors for Multi-view Correspondence
This work addresses the challenge of semantic correspondence in multi-view sketches for applications in computer vision and graphics, representing an incremental advancement with a novel method for a known bottleneck.
The paper tackles the problem of multi-view sketch correspondence by learning a novel local sketch descriptor using a deep learning approach, achieving state-of-the-art performance with a 15% improvement in accuracy over baseline methods on hand-drawn sketches.
In this paper, we study the problem of multi-view sketch correspondence, where we take as input multiple freehand sketches with different views of the same object and predict as output the semantic correspondence among the sketches. This problem is challenging since the visual features of corresponding points at different views can be very different. To this end, we take a deep learning approach and learn a novel local sketch descriptor from data. We contribute a training dataset by generating the pixel-level correspondence for the multi-view line drawings synthesized from 3D shapes. To handle the sparsity and ambiguity of sketches, we design a novel multi-branch neural network that integrates a patch-based representation and a multi-scale strategy to learn the pixel-level correspondence among multi-view sketches. We demonstrate the effectiveness of our proposed approach with extensive experiments on hand-drawn sketches and multi-view line drawings rendered from multiple 3D shape datasets.