Topological Data Analysis Guided Segment Anything Model Prompt Optimization for Zero-Shot Segmentation in Biological Imaging
This work addresses zero-shot segmentation challenges in biological imaging, offering an incremental improvement over existing methods.
The paper tackles the problem of optimizing prompts for the Segment Anything Model (SAM) in biological image segmentation by using topological data analysis (TDA) to guide point selection, resulting in improved performance for small objects and reduced computational complexity.
Emerging foundation models in machine learning are models trained on vast amounts of data that have been shown to generalize well to new tasks. Often these models can be prompted with multi-modal inputs that range from natural language descriptions over images to point clouds. In this paper, we propose topological data analysis (TDA) guided prompt optimization for the Segment Anything Model (SAM) and show preliminary results in the biological image segmentation domain. Our approach replaces the standard grid search approach that is used in the original implementation and finds point locations based on their topological significance. Our results show that the TDA optimized point cloud is much better suited for finding small objects and massively reduces computational complexity despite the extra step in scenarios which require many segmentations.