mustGAN: Multi-Stream Generative Adversarial Networks for MR Image Synthesis
This work addresses the limitation of multi-contrast MRI protocols in clinical settings by improving image synthesis, though it is incremental as it builds on existing one-to-one and many-to-one methods.
The paper tackles the problem of synthesizing missing or corrupted MRI contrasts by proposing a multi-stream GAN that combines one-to-one and many-to-one streams with adaptive fusion, achieving superior performance over previous state-of-the-art methods in qualitative and quantitative assessments on T1-, T2-, PD-weighted, and FLAIR images.
Multi-contrast MRI protocols increase the level of morphological information available for diagnosis. Yet, the number and quality of contrasts is limited in practice by various factors including scan time and patient motion. Synthesis of missing or corrupted contrasts can alleviate this limitation to improve clinical utility. Common approaches for multi-contrast MRI involve either one-to-one and many-to-one synthesis methods. One-to-one methods take as input a single source contrast, and they learn a latent representation sensitive to unique features of the source. Meanwhile, many-to-one methods receive multiple distinct sources, and they learn a shared latent representation more sensitive to common features across sources. For enhanced image synthesis, here we propose a multi-stream approach that aggregates information across multiple source images via a mixture of multiple one-to-one streams and a joint many-to-one stream. The shared feature maps generated in the many-to-one stream and the complementary feature maps generated in the one-to-one streams are combined with a fusion block. The location of the fusion block is adaptively modified to maximize task-specific performance. Qualitative and quantitative assessments on T1-, T2-, PD-weighted and FLAIR images clearly demonstrate the superior performance of the proposed method compared to previous state-of-the-art one-to-one and many-to-one methods.