UniverSeg: Universal Medical Image Segmentation
This addresses a barrier for clinical researchers who lack resources to train models for new segmentation tasks, though it is incremental as it builds on existing few-shot segmentation approaches.
The paper tackles the problem of medical image segmentation models failing to generalize to unseen tasks involving new anatomies or modalities, presenting UniverSeg which achieves accurate segmentation without additional training by using a Cross-Block mechanism and training on a diverse dataset of 53 datasets with over 22,000 scans, outperforming related methods on unseen tasks.
While deep learning models have become the predominant method for medical image segmentation, they are typically not capable of generalizing to unseen segmentation tasks involving new anatomies, image modalities, or labels. Given a new segmentation task, researchers generally have to train or fine-tune models, which is time-consuming and poses a substantial barrier for clinical researchers, who often lack the resources and expertise to train neural networks. We present UniverSeg, a method for solving unseen medical segmentation tasks without additional training. Given a query image and example set of image-label pairs that define a new segmentation task, UniverSeg employs a new Cross-Block mechanism to produce accurate segmentation maps without the need for additional training. To achieve generalization to new tasks, we have gathered and standardized a collection of 53 open-access medical segmentation datasets with over 22,000 scans, which we refer to as MegaMedical. We used this collection to train UniverSeg on a diverse set of anatomies and imaging modalities. We demonstrate that UniverSeg substantially outperforms several related methods on unseen tasks, and thoroughly analyze and draw insights about important aspects of the proposed system. The UniverSeg source code and model weights are freely available at https://universeg.csail.mit.edu