Leveraging Unlabeled Data for Sketch-based Understanding
This addresses the limitation of sketch-based models that rely heavily on labeled data, potentially enhancing applicability in domains like computer vision, though it appears incremental as it adapts existing methods to sketches.
The paper tackles the problem of sketch-based understanding by leveraging unlabeled data to improve model generalization, showing that their sketch-BYOL method outperforms other self-supervised approaches in retrieval performance for known and unknown categories.
Sketch-based understanding is a critical component of human cognitive learning and is a primitive communication means between humans. This topic has recently attracted the interest of the computer vision community as sketching represents a powerful tool to express static objects and dynamic scenes. Unfortunately, despite its broad application domains, the current sketch-based models strongly rely on labels for supervised training, ignoring knowledge from unlabeled data, thus limiting the underlying generalization and the applicability. Therefore, we present a study about the use of unlabeled data to improve a sketch-based model. To this end, we evaluate variations of VAE and semi-supervised VAE, and present an extension of BYOL to deal with sketches. Our results show the superiority of sketch-BYOL, which outperforms other self-supervised approaches increasing the retrieval performance for known and unknown categories. Furthermore, we show how other tasks can benefit from our proposal.