Learning Features and their Transformations by Spatial and Temporal Spherical Clustering
This work addresses the challenge of building recognition systems that are invariant to transformations, which is incremental as it builds on existing neural models but introduces a novel clustering approach.
The paper tackles the problem of learning features invariant to transformations in visual data by proposing a two-layered neural model that learns features via spatial spherical clustering and invariance via temporal spherical clustering in an unsupervised manner. When tested on natural videos from a cat-mounted camera, the model's neurons developed simple and complex cell-like receptive fields, with a topographic map emerging through lateral connections.
Learning features invariant to arbitrary transformations in the data is a requirement for any recognition system, biological or artificial. It is now widely accepted that simple cells in the primary visual cortex respond to features while the complex cells respond to features invariant to different transformations. We present a novel two-layered feedforward neural model that learns features in the first layer by spatial spherical clustering and invariance to transformations in the second layer by temporal spherical clustering. Learning occurs in an online and unsupervised manner following the Hebbian rule. When exposed to natural videos acquired by a camera mounted on a cat's head, the first and second layer neurons in our model develop simple and complex cell-like receptive field properties. The model can predict by learning lateral connections among the first layer neurons. A topographic map to their spatial features emerges by exponentially decaying the flow of activation with distance from one neuron to another in the first layer that fire in close temporal proximity, thereby minimizing the pooling length in an online manner simultaneously with feature learning.