SonoSAMTrack -- Segment and Track Anything on Ultrasound Images
This work provides a valuable tool for generating dense annotations and segmentations of anatomical structures in clinical ultrasound workflows, though it is incremental as it builds on existing foundational and tracking models.
The authors tackled the problem of segmenting and tracking objects in ultrasound images by combining a promptable foundational segmentation model (SonoSAM) with a contour tracking model, achieving state-of-the-art performance on 7 unseen datasets with significant margins over competitors and reducing clicks in dense video annotation tasks.
In this paper, we present SonoSAMTrack - that combines a promptable foundational model for segmenting objects of interest on ultrasound images called SonoSAM, with a state-of-the art contour tracking model to propagate segmentations on 2D+t and 3D ultrasound datasets. Fine-tuned and tested exclusively on a rich, diverse set of objects from $\approx200$k ultrasound image-mask pairs, SonoSAM demonstrates state-of-the-art performance on 7 unseen ultrasound data-sets, outperforming competing methods by a significant margin. We also extend SonoSAM to 2-D +t applications and demonstrate superior performance making it a valuable tool for generating dense annotations and segmentation of anatomical structures in clinical workflows. Further, to increase practical utility of the work, we propose a two-step process of fine-tuning followed by knowledge distillation to a smaller footprint model without comprising the performance. We present detailed qualitative and quantitative comparisons of SonoSAM with state-of-the-art methods showcasing efficacy of the method. This is followed by demonstrating the reduction in number of clicks in a dense video annotation problem of adult cardiac ultrasound chamber segmentation using SonoSAMTrack.