Multiscale Voxel Based Decoding For Enhanced Natural Image Reconstruction From Brain Activity
This work addresses the challenge of enhancing image reconstruction from brain activity for applications in neuroscience and brain-computer interfaces, but it appears incremental as it combines existing techniques.
The study tackled the problem of reconstructing natural images from fMRI brain activity, which often yields blurry results, by merging object decoding and image reconstruction methods using a class-conditional GAN and neural style transfer, leading to improved semantic similarity in the reconstructed images.
Reconstructing perceived images from human brain activity monitored by functional magnetic resonance imaging (fMRI) is hard, especially for natural images. Existing methods often result in blurry and unintelligible reconstructions with low fidelity. In this study, we present a novel approach for enhanced image reconstruction, in which existing methods for object decoding and image reconstruction are merged together. This is achieved by conditioning the reconstructed image to its decoded image category using a class-conditional generative adversarial network and neural style transfer. The results indicate that our approach improves the semantic similarity of the reconstructed images and can be used as a general framework for enhanced image reconstruction.