Zero-Shot Sketch-Based Image Retrieval with Structure-aware Asymmetric Disentanglement
This addresses a practical limitation in real-world sketch-based image retrieval by enabling retrieval for unseen categories, though it is an incremental advancement in domain adaptation for retrieval tasks.
The paper tackles the problem of Zero-Shot Sketch-Based Image Retrieval (ZS-SBIR), where test categories are unseen during training, by proposing the STRAD method that disentangles image features into structure and appearance components while projecting sketches to structure space. The method achieves state-of-the-art performance, with experiments showing remarkable improvements on three large-scale benchmark datasets.
The goal of Sketch-Based Image Retrieval (SBIR) is using free-hand sketches to retrieve images of the same category from a natural image gallery. However, SBIR requires all test categories to be seen during training, which cannot be guaranteed in real-world applications. So we investigate more challenging Zero-Shot SBIR (ZS-SBIR), in which test categories do not appear in the training stage. After realizing that sketches mainly contain structure information while images contain additional appearance information, we attempt to achieve structure-aware retrieval via asymmetric disentanglement.For this purpose, we propose our STRucture-aware Asymmetric Disentanglement (STRAD) method, in which image features are disentangled into structure features and appearance features while sketch features are only projected to structure space. Through disentangling structure and appearance space, bi-directional domain translation is performed between the sketch domain and the image domain. Extensive experiments demonstrate that our STRAD method remarkably outperforms state-of-the-art methods on three large-scale benchmark datasets.