Online Representation Learning with Single and Multi-layer Hebbian Networks for Image Classification
This work addresses the problem of developing compact, online learning networks for image classification, but it is incremental as it builds on existing Hebbian-like methods.
The study tackled unsupervised feature learning from images using a Hebbian-like rule derived from a nonnegative classical multidimensional scaling cost-function, applied to single and multi-layer networks, and achieved competitive performance on CIFAR-10 classification when features were used with an SVM.
Unsupervised learning permits the development of algorithms that are able to adapt to a variety of different data sets using the same underlying rules thanks to the autonomous discovery of discriminating features during training. Recently, a new class of Hebbian-like and local unsupervised learning rules for neural networks have been developed that minimise a similarity matching cost-function. These have been shown to perform sparse representation learning. This study tests the effectiveness of one such learning rule for learning features from images. The rule implemented is derived from a nonnegative classical multidimensional scaling cost-function, and is applied to both single and multi-layer architectures. The features learned by the algorithm are then used as input to an SVM to test their effectiveness in classification on the established CIFAR-10 image dataset. The algorithm performs well in comparison to other unsupervised learning algorithms and multi-layer networks, thus suggesting its validity in the design of a new class of compact, online learning networks.