Colony Grounded SAM2: Zero-shot detection and segmentation of bacterial colonies using foundation models
This work addresses the lack of labeled datasets for bacterial colony analysis in microbiology, though it is incremental as it adapts existing foundation models to a specific domain.
The paper tackled the problem of detecting and segmenting bacterial colonies in agar-plate images without labeled datasets by proposing Colony Grounded SAM2, a zero-shot inference pipeline using pre-trained foundation models, achieving a mean Average Precision of 93.1% and a Dice@detection score of 0.85 on out-of-distribution data.
The detection and classification of bacterial colonies in images of agar-plates is important in microbiology, but is hindered by the lack of labeled datasets. Therefore, we propose Colony Grounded SAM2, a zero-shot inference pipeline to detect and segment bacterial colonies in multiple settings without any further training. By utilizing the pre-trained foundation models Grounding DINO and Segment Anything Model 2, fine-tuned to the microbiological domain, we developed a model that is robust to data changes. Results showed a mean Average Precision of 93.1\% and a $Dice@detection$ score of 0.85, showing excellent detection and segmentation capabilities on out-of-distribution datasets. The entire pipeline with model weights are shared open access to aid with annotation- and classification purposes in microbiology.