Unlocking Zero-Shot Plant Segmentation with Pl@ntNet Intelligence
This addresses the annotation bottleneck for plant segmentation in diverse agricultural scenarios, though it's an incremental combination of existing foundation models.
The researchers tackled plant segmentation in agricultural imagery without requiring new annotated datasets by combining Plantnet's specialized plant representations with SAM for refinement. Their approach achieved consistent performance gains over baseline methods, with improvements measured by Jaccard Index (IoU) across four datasets of varying complexity.
We present a zero-shot segmentation approach for agricultural imagery that leverages Plantnet, a large-scale plant classification model, in conjunction with its DinoV2 backbone and the Segment Anything Model (SAM). Rather than collecting and annotating new datasets, our method exploits Plantnet's specialized plant representations to identify plant regions and produce coarse segmentation masks. These masks are then refined by SAM to yield detailed segmentations. We evaluate on four publicly available datasets of various complexity in terms of contrast including some where the limited size of the training data and complex field conditions often hinder purely supervised methods. Our results show consistent performance gains when using Plantnet-fine-tuned DinoV2 over the base DinoV2 model, as measured by the Jaccard Index (IoU). These findings highlight the potential of combining foundation models with specialized plant-centric models to alleviate the annotation bottleneck and enable effective segmentation in diverse agricultural scenarios.