Alzheimer's Disease Prediction via Brain Structural-Functional Deep Fusing Network
This work addresses Alzheimer's disease diagnosis for medical researchers and clinicians, offering an incremental improvement in multimodal neuroimage fusion.
The paper tackled the challenge of fusing structural and functional brain images for Alzheimer's disease prediction by proposing a cross-modal transformer generative adversarial network (CT-GAN), which improved prediction performance on the ADNI dataset and effectively identified AD-related brain connections.
Fusing structural-functional images of the brain has shown great potential to analyze the deterioration of Alzheimer's disease (AD). However, it is a big challenge to effectively fuse the correlated and complementary information from multimodal neuroimages. In this paper, a novel model termed cross-modal transformer generative adversarial network (CT-GAN) is proposed to effectively fuse the functional and structural information contained in functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). The CT-GAN can learn topological features and generate multimodal connectivity from multimodal imaging data in an efficient end-to-end manner. Moreover, the swapping bi-attention mechanism is designed to gradually align common features and effectively enhance the complementary features between modalities. By analyzing the generated connectivity features, the proposed model can identify AD-related brain connections. Evaluations on the public ADNI dataset show that the proposed CT-GAN can dramatically improve prediction performance and detect AD-related brain regions effectively. The proposed model also provides new insights for detecting AD-related abnormal neural circuits.