Characterising representation dynamics in recurrent neural networks for object recognition
This work addresses the problem of understanding recurrent computation dynamics in vision models for researchers in computational neuroscience and AI, though it is incremental in characterizing existing RNN behaviors.
The study investigated how representations evolve in recurrent neural networks (RNNs) trained for object recognition on MiniEcoset, finding that representations continue to change after correct classification and that misclassified ones have lower L2 norms and peripheral positions, aiding correction over time.
Recurrent neural networks (RNNs) have yielded promising results for both recognizing objects in challenging conditions and modeling aspects of primate vision. However, the representational dynamics of recurrent computations remain poorly understood, especially in large-scale visual models. Here, we studied such dynamics in RNNs trained for object classification on MiniEcoset, a novel subset of ecoset. We report two main insights. First, upon inference, representations continued to evolve after correct classification, suggesting a lack of the notion of being ``done with classification''. Second, focusing on ``readout zones'' as a way to characterize the activation trajectories, we observe that misclassified representations exhibit activation patterns with lower L2 norm, and are positioned more peripherally in the readout zones. Such arrangements help the misclassified representations move into the correct zones as time progresses. Our findings generalize to networks with lateral and top-down connections, and include both additive and multiplicative interactions with the bottom-up sweep. The results therefore contribute to a general understanding of RNN dynamics in naturalistic tasks. We hope that the analysis framework will aid future investigations of other types of RNNs, including understanding of representational dynamics in primate vision.