Latent graph encoding of multimodal neuroimaging features with generative AI architectures
For neuroimaging researchers, this work provides a systematic evaluation and a novel graph-based VAE architecture that improves multimodal feature encoding and generation.
The paper designs a multimodal generative framework for structural and functional MRI features, evaluating VAEs, transformers, GANs, and diffusion models. The proposed graph VAE (gMMVAE) outperforms alternatives in generation fidelity, reconstruction quality, efficiency, and latent space discriminability.
While generative models enable encoding of complex neuroimaging data for feature generation and reconstruction, developing optimal architectural frameworks with appropriate encoding and latent space processes is crucial for studying structural and functional properties of the brain. We design a multimodal generative framework for structural and functional magnetic resonance imaging (MRI) features through systematic evaluation of encoding strategies, latent multimodal fusion, and generative model selection. Using structural gray matter volume (GMV) and static functional network connectivity (sFNC) features from a large neuroimaging dataset, we analyze generative frameworks involving variational autoencoders (VAEs), transformers, generative adversarial networks (GANs), and diffusion models. Architectures that employ modality-aware graph encoding of functional connectivity into a lower-dimensional latent space outperform vectorized encoders or direct data space approaches. The proposed multimodal graph VAE (gMMVAE) surpasses alternative generative variants across multiple metrics for generation fidelity, reconstruction quality, efficiency, and latent space discriminability, highlighting its potential for robust multimodal neuroimaging analysis.