SEMPART: Self-supervised Multi-resolution Partitioning of Image Semantics
This work addresses the challenge of unsupervised object partitioning in computer vision, offering an incremental improvement over existing self-supervised methods.
The paper tackles the problem of accurately determining salient regions in images with scarce labeled data by proposing SEMPART, a method that jointly infers coarse and fine bi-partitions over a DINO-based semantic graph, resulting in high-quality masks rapidly without post-processing and showing benefits from co-optimizing branches.
Accurately determining salient regions of an image is challenging when labeled data is scarce. DINO-based self-supervised approaches have recently leveraged meaningful image semantics captured by patch-wise features for locating foreground objects. Recent methods have also incorporated intuitive priors and demonstrated value in unsupervised methods for object partitioning. In this paper, we propose SEMPART, which jointly infers coarse and fine bi-partitions over an image's DINO-based semantic graph. Furthermore, SEMPART preserves fine boundary details using graph-driven regularization and successfully distills the coarse mask semantics into the fine mask. Our salient object detection and single object localization findings suggest that SEMPART produces high-quality masks rapidly without additional post-processing and benefits from co-optimizing the coarse and fine branches.