SAMReg: SAM-enabled Image Registration with ROI-based Correspondence
This addresses medical image registration for clinical applications by introducing a novel correspondence representation that bridges registration and segmentation, though it appears incremental as it builds on existing foundation models.
The paper tackles medical image registration by proposing a new ROI-based correspondence representation and developing SAMReg, a registration algorithm based on the Segment Anything Model that requires no training or fine-tuning. The method outperforms intensity-based iterative algorithms and DDF-predicting learning-based networks across five real-world applications, achieving improvements in Dice scores and target registration errors.
This paper describes a new spatial correspondence representation based on paired regions-of-interest (ROIs), for medical image registration. The distinct properties of the proposed ROI-based correspondence are discussed, in the context of potential benefits in clinical applications following image registration, compared with alternative correspondence-representing approaches, such as those based on sampled displacements and spatial transformation functions. These benefits include a clear connection between learning-based image registration and segmentation, which in turn motivates two cases of image registration approaches using (pre-)trained segmentation networks. Based on the segment anything model (SAM), a vision foundation model for segmentation, we develop a new registration algorithm SAMReg, which does not require any training (or training data), gradient-based fine-tuning or prompt engineering. The proposed SAMReg models are evaluated across five real-world applications, including intra-subject registration tasks with cardiac MR and lung CT, challenging inter-subject registration scenarios with prostate MR and retinal imaging, and an additional evaluation with a non-clinical example with aerial image registration. The proposed methods outperform both intensity-based iterative algorithms and DDF-predicting learning-based networks across tested metrics including Dice and target registration errors on anatomical structures, and further demonstrates competitive performance compared to weakly-supervised registration approaches that rely on fully-segmented training data. Open source code and examples are available at: https://github.com/sqhuang0103/SAMReg.git.