Weakly Supervised Panoptic Segmentation for Defect-Based Grading of Fresh Produce
This work addresses the challenge of labor-intensive and error-prone visual inspection for defect grading in decentralized agricultural supply chains, offering an incremental improvement by leveraging foundation models in low-data scenarios.
The paper tackled the problem of automating defect grading in agricultural supply chains by using the Segment Anything Model (SAM) to generate dense panoptic segmentation masks from sparse annotations, reducing annotation effort and providing practical estimates of defect number and relative size, validated on 476 field images with 1440 defects.
Visual inspection for defect grading in agricultural supply chains is crucial but traditionally labor-intensive and error-prone. Automated computer vision methods typically require extensively annotated datasets, which are often unavailable in decentralized supply chains. We address this challenge by evaluating the Segment Anything Model (SAM) to generate dense panoptic segmentation masks from sparse annotations. These dense predictions are then used to train a supervised panoptic segmentation model. Focusing on banana surface defects (bruises and scars), we validate our approach using 476 field images annotated with 1440 defects. While SAM-generated masks generally align with human annotations, substantially reducing annotation effort, we explicitly identify failure cases associated with specific defect sizes and shapes. Despite these limitations, our approach offers practical estimates of defect number and relative size from panoptic masks, underscoring the potential and current boundaries of foundation models for defect quantification in low-data agricultural scenarios. GitHub: https://github.com/manuelknott/banana-defect-segmentation