Adaptive Interactive Segmentation for Multimodal Medical Imaging via Selection Engine
This addresses segmentation challenges for medical image analysis, offering improved adaptability and generalization, though it appears incremental as it builds on SAM2 with enhancements for multimodal data.
The paper tackled the problem of fast and accurate segmentation in multimodal medical imaging by proposing the SISeg model with an Adaptive Frame Selection Engine, achieving robust adaptability and generalization across 10 datasets covering 7 modalities.
In medical image analysis, achieving fast, efficient, and accurate segmentation is essential for automated diagnosis and treatment. Although recent advancements in deep learning have significantly improved segmentation accuracy, current models often face challenges in adaptability and generalization, particularly when processing multi-modal medical imaging data. These limitations stem from the substantial variations between imaging modalities and the inherent complexity of medical data. To address these challenges, we propose the Strategy-driven Interactive Segmentation Model (SISeg), built on SAM2, which enhances segmentation performance across various medical imaging modalities by integrating a selection engine. To mitigate memory bottlenecks and optimize prompt frame selection during the inference of 2D image sequences, we developed an automated system, the Adaptive Frame Selection Engine (AFSE). This system dynamically selects the optimal prompt frames without requiring extensive prior medical knowledge and enhances the interpretability of the model's inference process through an interactive feedback mechanism. We conducted extensive experiments on 10 datasets covering 7 representative medical imaging modalities, demonstrating the SISeg model's robust adaptability and generalization in multi-modal tasks. The project page and code will be available at: [URL].