Cross-Modality Neuroimage Synthesis: A Survey
This is an incremental survey paper that synthesizes existing research for researchers in medical imaging and neuroinformatics.
This paper provides a comprehensive survey of cross-modality neuroimage synthesis methods, addressing the challenge of synthesizing missing neuroimaging data due to difficulties in collecting fully aligned and paired multi-modality data, such as high cost and privacy issues. It reviews techniques from weakly supervised and unsupervised settings, covering aspects like loss functions, evaluation metrics, and downstream applications.
Multi-modality imaging improves disease diagnosis and reveals distinct deviations in tissues with anatomical properties. The existence of completely aligned and paired multi-modality neuroimaging data has proved its effectiveness in brain research. However, collecting fully aligned and paired data is expensive or even impractical, since it faces many difficulties, including high cost, long acquisition time, image corruption, and privacy issues. An alternative solution is to explore unsupervised or weakly supervised learning methods to synthesize the absent neuroimaging data. In this paper, we provide a comprehensive review of cross-modality synthesis for neuroimages, from the perspectives of weakly supervised and unsupervised settings, loss functions, evaluation metrics, imaging modalities, datasets, and downstream applications based on synthesis. We begin by highlighting several opening challenges for cross-modality neuroimage synthesis. Then, we discuss representative architectures of cross-modality synthesis methods under different supervisions. This is followed by a stepwise in-depth analysis to evaluate how cross-modality neuroimage synthesis improves the performance of its downstream tasks. Finally, we summarize the existing research findings and point out future research directions. All resources are available at https://github.com/M-3LAB/awesome-multimodal-brain-image-systhesis