SAM++: Enhancing Anatomic Matching using Semantic Information and Structural Inference
This work addresses a domain-specific problem in medical imaging for clinicians, offering incremental advancements over prior methods.
The paper tackles the problem of accurately matching anatomical structures in medical images like CT and MRI, which is crucial for clinical workflows, by proposing SAM++, a framework that learns appearance and semantic embeddings with a fixed-points matching mechanism, resulting in significant improvements over existing methods including SAM.
Medical images like CT and MRI provide detailed information about the internal structure of the body, and identifying key anatomical structures from these images plays a crucial role in clinical workflows. Current methods treat it as a registration or key-point regression task, which has limitations in accurate matching and can only handle predefined landmarks. Recently, some methods have been introduced to address these limitations. One such method, called SAM, proposes using a dense self-supervised approach to learn a distinct embedding for each point on the CT image and achieving promising results. Nonetheless, SAM may still face difficulties when dealing with structures that have similar appearances but different semantic meanings or similar semantic meanings but different appearances. To overcome these limitations, we propose SAM++, a framework that simultaneously learns appearance and semantic embeddings with a novel fixed-points matching mechanism. We tested the SAM++ framework on two challenging tasks, demonstrating a significant improvement over the performance of SAM and outperforming other existing methods.