NEvo: Neural-Guided Evolutionary Video Synthesis for Dynamic Visual Selectivity
For neuroscientists studying dynamic visual processing, this framework enables in silico exploration of selectivity in brain regions that were previously limited to static images, offering new predictions for in vivo experiments.
The paper introduces NEvo, a neural-guided video synthesis framework that generates dynamic stimuli optimized for specific brain regions, outperforming handcrafted localizer videos in activating target ROIs across visual cortex.
The human brain processes dynamic visual input through hierarchically organized, functionally specialized regions. While recent in silico brain encoding models can synthesize optimal stimuli to probe selectivity in different brain regions, prior work has been largely limited to static images, leaving dynamic visual processing underexplored. We introduce a novel neural-guided video synthesis framework that generates stimuli optimized for target brain regions across visual cortex. Our method performs evolutionary search over a structured prompt space, guided by a dynamic encoding model that predicts voxel-level responses to video inputs. By maximizing predicted activity for a target ROI, the framework efficiently discovers hyper-activating dynamic stimuli that consistently surpass handcrafted localizer videos. The synthesized videos recover known selectivities across ventral, dorsal, and lateral pathways, and further reveal systematic differences in sensitivity to temporal dynamics. A searchlight analysis provides new insight into the progression toward increasingly complex social-dynamic features along the lateral stream, further supported by probing with synthesized abstract, non-naturalistic stimuli. Taken together, our framework enables in silico exploration of dynamic visual selectivity, with new predictions for in vivo experiments