Learning V1 Simple Cells with Vector Representation of Local Content and Matrix Representation of Local Motion
This work addresses motion perception in primary visual cortex for neuroscience and computer vision, though it appears incremental as it builds on existing models with a specific coupling approach.
The paper tackles the problem of modeling motion perception in V1 by proposing a representational model for image pairs that couples vector representations of local content with matrix representations of local pixel displacements, resulting in learned Gabor-like filter pairs matching Macaque V1 simple cells and competitive optical flow estimation.
This paper proposes a representational model for image pairs such as consecutive video frames that are related by local pixel displacements, in the hope that the model may shed light on motion perception in primary visual cortex (V1). The model couples the following two components: (1) the vector representations of local contents of images and (2) the matrix representations of local pixel displacements caused by the relative motions between the agent and the objects in the 3D scene. When the image frame undergoes changes due to local pixel displacements, the vectors are multiplied by the matrices that represent the local displacements. Thus the vector representation is equivariant as it varies according to the local displacements. Our experiments show that our model can learn Gabor-like filter pairs of quadrature phases. The profiles of the learned filters match those of simple cells in Macaque V1. Moreover, we demonstrate that the model can learn to infer local motions in either a supervised or unsupervised manner. With such a simple model, we achieve competitive results on optical flow estimation.