Optimizing deep video representation to match brain activity
This work addresses the problem of probing brain activity under ecological conditions for neuroscience researchers, but it is incremental as it applies existing deep learning methods to new fMRI data.
The study tackled the problem of understanding brain functional organization by comparing fMRI activity from subjects watching natural movies with deep video representations using optical flow and image content. The result revealed complexity-related contrasts across visual areas and a foveal/peripheral dichotomy.
The comparison of observed brain activity with the statistics generated by artificial intelligence systems is useful to probe brain functional organization under ecological conditions. Here we study fMRI activity in ten subjects watching color natural movies and compute deep representations of these movies with an architecture that relies on optical flow and image content. The association of activity in visual areas with the different layers of the deep architecture displays complexity-related contrasts across visual areas and reveals a striking foveal/peripheral dichotomy.