Active Learning for Fine-Grained Sketch-Based Image Retrieval
This work addresses the scalability and practical adoption challenges in fine-grained sketch-based image retrieval by reducing labeling costs, though it is incremental as it applies active learning to a new domain.
The paper tackles the problem of reducing the sketching effort needed for fine-grained sketch-based image retrieval by proposing a novel active learning sampling technique that minimizes the need for labeled sketches, achieving superior performance over baselines on ChairV2 and ShoeV2 datasets.
The ability to retrieve a photo by mere free-hand sketching highlights the immense potential of Fine-grained sketch-based image retrieval (FG-SBIR). However, its rapid practical adoption, as well as scalability, is limited by the expense of acquiring faithful sketches for easily available photo counterparts. A solution to this problem is Active Learning, which could minimise the need for labeled sketches while maximising performance. Despite extensive studies in the field, there exists no work that utilises it for reducing sketching effort in FG-SBIR tasks. To this end, we propose a novel active learning sampling technique that drastically minimises the need for drawing photo sketches. Our proposed approach tackles the trade-off between uncertainty and diversity by utilising the relationship between the existing photo-sketch pair to a photo that does not have its sketch and augmenting this relation with its intermediate representations. Since our approach relies only on the underlying data distribution, it is agnostic of the modelling approach and hence is applicable to other cross-modal instance-level retrieval tasks as well. With experimentation over two publicly available fine-grained SBIR datasets ChairV2 and ShoeV2, we validate our approach and reveal its superiority over adapted baselines.