Large-Scale Feature Learning With Spike-and-Slab Sparse Coding
This addresses the problem of limited labeled data in large-scale object recognition for computer vision researchers, though it is incremental as it builds on prior S3C work.
The authors tackled object recognition with many classes by introducing a scalable feature learning method called spike-and-slab sparse coding (S3C), which improved supervised learning on CIFAR-10 and scaled better on CIFAR-100, winning a NIPS 2011 challenge.
We consider the problem of object recognition with a large number of classes. In order to overcome the low amount of labeled examples available in this setting, we introduce a new feature learning and extraction procedure based on a factor model we call spike-and-slab sparse coding (S3C). Prior work on S3C has not prioritized the ability to exploit parallel architectures and scale S3C to the enormous problem sizes needed for object recognition. We present a novel inference procedure for appropriate for use with GPUs which allows us to dramatically increase both the training set size and the amount of latent factors that S3C may be trained with. We demonstrate that this approach improves upon the supervised learning capabilities of both sparse coding and the spike-and-slab Restricted Boltzmann Machine (ssRBM) on the CIFAR-10 dataset. We use the CIFAR-100 dataset to demonstrate that our method scales to large numbers of classes better than previous methods. Finally, we use our method to win the NIPS 2011 Workshop on Challenges In Learning Hierarchical Models? Transfer Learning Challenge.