MedPrompt: Cross-Modal Prompting for Multi-Task Medical Image Translation
This work addresses the need for versatile medical image translation to synthesize missing modality data for clinical diagnosis, representing an incremental improvement over prior methods by enhancing cross-modal feature capture.
The paper tackles the problem of cross-modal medical image translation, where existing methods are limited to specific modality pairs, by introducing MedPrompt, a multi-task framework that dynamically adapts to different modalities using a Self-adaptive Prompt Block and Transformer for global features, achieving state-of-the-art visual quality and generalization across five datasets and four modality pairs.
Cross-modal medical image translation is an essential task for synthesizing missing modality data for clinical diagnosis. However, current learning-based techniques have limitations in capturing cross-modal and global features, restricting their suitability to specific pairs of modalities. This lack of versatility undermines their practical usefulness, particularly considering that the missing modality may vary for different cases. In this study, we present MedPrompt, a multi-task framework that efficiently translates different modalities. Specifically, we propose the Self-adaptive Prompt Block, which dynamically guides the translation network towards distinct modalities. Within this framework, we introduce the Prompt Extraction Block and the Prompt Fusion Block to efficiently encode the cross-modal prompt. To enhance the extraction of global features across diverse modalities, we incorporate the Transformer model. Extensive experimental results involving five datasets and four pairs of modalities demonstrate that our proposed model achieves state-of-the-art visual quality and exhibits excellent generalization capability.