PlantSAM: An Object Detection-Driven Segmentation Pipeline for Herbarium Specimens
This addresses background heterogeneity in herbarium images for botanical researchers, though it is incremental as it combines existing models.
The paper tackled background noise in herbarium image classification by developing PlantSAM, an automated segmentation pipeline combining YOLOv10 and SAM2, which achieved state-of-the-art segmentation performance with an IoU of 0.94 and Dice coefficient of 0.97, leading to classification accuracy gains of up to 4.36%.
Deep learning-based classification of herbarium images is hampered by background heterogeneity, which introduces noise and artifacts that can potentially mislead models and reduce classification accuracy. Addressing these background-related challenges is critical to improving model performance. We introduce PlantSAM, an automated segmentation pipeline that integrates YOLOv10 for plant region detection and the Segment Anything Model (SAM2) for segmentation. YOLOv10 generates bounding box prompts to guide SAM2, enhancing segmentation accuracy. Both models were fine-tuned on herbarium images and evaluated using Intersection over Union (IoU) and Dice coefficient metrics. PlantSAM achieved state-of-the-art segmentation performance, with an IoU of 0.94 and a Dice coefficient of 0.97. Incorporating segmented images into classification models led to consistent performance improvements across five tested botanical traits, with accuracy gains of up to 4.36% and F1-score improvements of 4.15%. Our findings highlight the importance of background removal in herbarium image analysis, as it significantly enhances classification accuracy by allowing models to focus more effectively on the foreground plant structures.