Ensemble manifold based regularized multi-modal graph convolutional network for cognitive ability prediction
This work addresses the problem of improving cognitive ability prediction for neuroscience researchers by developing an interpretable deep learning framework, though it appears incremental as it builds on existing graph convolutional networks with multi-modal fusion.
The paper tackled predicting cognitive ability from multi-modal fMRI data by proposing an interpretable multi-modal graph convolutional network (MGCN) that integrates time series and functional connectivity, achieving superior predictive performance over single-modality GCN and other methods on the Philadelphia Neurodevelopmental Cohort for WRAT score prediction.
Objective: Multi-modal functional magnetic resonance imaging (fMRI) can be used to make predictions about individual behavioral and cognitive traits based on brain connectivity networks. Methods: To take advantage of complementary information from multi-modal fMRI, we propose an interpretable multi-modal graph convolutional network (MGCN) model, incorporating the fMRI time series and the functional connectivity (FC) between each pair of brain regions. Specifically, our model learns a graph embedding from individual brain networks derived from multi-modal data. A manifold-based regularization term is then enforced to consider the relationships of subjects both within and between modalities. Furthermore, we propose the gradient-weighted regression activation mapping (Grad-RAM) and the edge mask learning to interpret the model, which is used to identify significant cognition-related biomarkers. Results: We validate our MGCN model on the Philadelphia Neurodevelopmental Cohort to predict individual wide range achievement test (WRAT) score. Our model obtains superior predictive performance over GCN with a single modality and other competing approaches. The identified biomarkers are cross-validated from different approaches. Conclusion and Significance: This paper develops a new interpretable graph deep learning framework for cognitive ability prediction, with the potential to overcome the limitations of several current data-fusion models. The results demonstrate the power of MGCN in analyzing multi-modal fMRI and discovering significant biomarkers for human brain studies.