Attention-Based Variational Framework for Joint and Individual Components Learning with Applications in Brain Network Analysis
This work addresses the challenge of multimodal brain network analysis for neuroscience researchers, offering an incremental improvement in disentangling shared and modality-specific information.
The paper tackled the problem of integrating structural and functional brain connectivity data by proposing CM-JIVNet, a probabilistic framework that learns factorized latent representations, resulting in superior performance in cross-modal reconstruction and behavioral trait prediction on HCP-YA data.
Brain organization is increasingly characterized through multiple imaging modalities, most notably structural connectivity (SC) and functional connectivity (FC). Integrating these inherently distinct yet complementary data sources is essential for uncovering the cross-modal patterns that drive behavioral phenotypes. However, effective integration is hindered by the high dimensionality and non-linearity of connectome data, complex non-linear SC-FC coupling, and the challenge of disentangling shared information from modality-specific variations. To address these issues, we propose the Cross-Modal Joint-Individual Variational Network (CM-JIVNet), a unified probabilistic framework designed to learn factorized latent representations from paired SC-FC datasets. Our model utilizes a multi-head attention fusion module to capture non-linear cross-modal dependencies while isolating independent, modality-specific signals. Validated on Human Connectome Project Young Adult (HCP-YA) data, CM-JIVNet demonstrates superior performance in cross-modal reconstruction and behavioral trait prediction. By effectively disentangling joint and individual feature spaces, CM-JIVNet provides a robust, interpretable, and scalable solution for large-scale multimodal brain analysis.