Cross-modal Subspace Learning for Fine-grained Sketch-based Image Retrieval
This work addresses the challenge of retrieving images from sketches for computer vision applications, but it is incremental as it benchmarks existing methods on new data.
The paper tackled the problem of sketch-based image retrieval by applying cross-modal subspace learning methods to bridge the domain gap between sketches and photos, demonstrating their effectiveness on fine-grained datasets.
Sketch-based image retrieval (SBIR) is challenging due to the inherent domain-gap between sketch and photo. Compared with pixel-perfect depictions of photos, sketches are iconic renderings of the real world with highly abstract. Therefore, matching sketch and photo directly using low-level visual clues are unsufficient, since a common low-level subspace that traverses semantically across the two modalities is non-trivial to establish. Most existing SBIR studies do not directly tackle this cross-modal problem. This naturally motivates us to explore the effectiveness of cross-modal retrieval methods in SBIR, which have been applied in the image-text matching successfully. In this paper, we introduce and compare a series of state-of-the-art cross-modal subspace learning methods and benchmark them on two recently released fine-grained SBIR datasets. Through thorough examination of the experimental results, we have demonstrated that the subspace learning can effectively model the sketch-photo domain-gap. In addition we draw a few key insights to drive future research.