DCNN-GAN: Reconstructing Realistic Image from fMRI
This work addresses the problem of visualizing perceptual content from fMRI for neuroscience and medical imaging, but it appears incremental as it builds on existing CNN and GAN techniques.
The paper tackled the challenge of reconstructing realistic images from fMRI data by proposing DCNN-GAN, a model combining a reconstruction network and GAN, which outperformed previous methods in reconstruction quality and computational cost.
Visualizing the perceptual content by analyzing human functional magnetic resonance imaging (fMRI) has been an active research area. However, due to its high dimensionality, complex dimensional structure, and small number of samples available, reconstructing realistic images from fMRI remains challenging. Recently with the development of convolutional neural network (CNN) and generative adversarial network (GAN), mapping multi-voxel fMRI data to complex, realistic images has been made possible. In this paper, we propose a model, DCNN-GAN, by combining a reconstruction network and GAN. We utilize the CNN for hierarchical feature extraction and the DCNN-GAN to reconstruct more realistic images. Extensive experiments have been conducted, showing that our method outperforms previous works, regarding reconstruction quality and computational cost.