Modeling Dynamic Neural Activity by combining Naturalistic Video Stimuli and Stimulus-independent Latent Factors
This work addresses a gap in dynamic neural encoding models for visual processing, offering a method to uncover biologically meaningful structure without explicit behavioral annotations, which is incremental but useful for neuroscience research.
The authors tackled the problem of modeling neural activity influenced by both external video stimuli and internal brain states by proposing a probabilistic model that predicts the joint distribution of neuronal responses. They found that the model outperformed video-only models in log-likelihood and correlation, with learned latent factors correlating strongly with mouse behavior and cortical patterns.
The neural activity in the visual processing is influenced by both external stimuli and internal brain states. Ideally, a neural predictive model should account for both of them. Currently, there are no dynamic encoding models that explicitly model a latent state and the entire neuronal response distribution. We address this gap by proposing a probabilistic model that predicts the joint distribution of the neuronal responses from video stimuli and stimulus-independent latent factors. After training and testing our model on mouse V1 neuronal responses, we find that it outperforms video-only models in terms of log-likelihood and achieves improvements in likelihood and correlation when conditioned on responses from other neurons. Furthermore, we find that the learned latent factors strongly correlate with mouse behavior and that they exhibit patterns related to the neurons' position on the visual cortex, although the model was trained without behavior and cortical coordinates. Our findings demonstrate that unsupervised learning of latent factors from population responses can reveal biologically meaningful structure that bridges sensory processing and behavior, without requiring explicit behavioral annotations during training.