Training-Free Fine-Grained Semantic Segmentations in Low Data Regimes: A FungiTastic Baseline
This work provides a scalable baseline for fine-grained semantic segmentation in low-data settings, addressing the problem of classifying visually similar classes with limited annotations.
The paper tackles fine-grained semantic segmentation in low-data regimes with a long-tailed distribution and varying acquisition conditions, as exemplified by the FungiTastic dataset. The proposed training-free two-stage framework (SAM3 for segmentation, DINOv3 with feature space transformation for classification) achieves the first baseline for this task, with results reported from one-shot to few-hundred-shot regimes.
Fine-grained semantic segmentation requires both precise localization and discrimination between visually similar classes. In FungiTastic, this problem is further complicated by a long-tailed distribution and strong variation in image acquisition conditions. We propose a training-free two-stage framework that decouples segmentation from classification. SAM3 first produces class-agnostic mushroom masks using macro-taxonomic prompts, and DINOv3 then assigns fine-grained labels through prototype matching in the embedding space. To improve this stage, we apply a simple transformation of the DINOv3 feature space that improves prototype-based classification. Compared with class-specific prompting, our approach is more scalable and keeps the segmentation cost low. We report results from one-shot to few-hundred-shot regimes, providing, to the best of our knowledge, the first baseline for fine-grained semantic segmentation in low-data settings.