Learning Invariant Representations with Local Transformations
This work addresses the need for robust feature learning that is invariant to transformations, benefiting domains like image and speech recognition, though it is incremental as it builds on existing unsupervised methods.
The paper tackles the problem of learning invariant representations in machine learning by introducing a framework that incorporates linear transformations into feature learning algorithms, such as transformation-invariant restricted Boltzmann machines, achieving competitive or superior performance on image classification benchmarks like MNIST variations, CIFAR-10, and STL-10, and state-of-the-art results on phone classification with the TIMIT dataset.
Learning invariant representations is an important problem in machine learning and pattern recognition. In this paper, we present a novel framework of transformation-invariant feature learning by incorporating linear transformations into the feature learning algorithms. For example, we present the transformation-invariant restricted Boltzmann machine that compactly represents data by its weights and their transformations, which achieves invariance of the feature representation via probabilistic max pooling. In addition, we show that our transformation-invariant feature learning framework can also be extended to other unsupervised learning methods, such as autoencoders or sparse coding. We evaluate our method on several image classification benchmark datasets, such as MNIST variations, CIFAR-10, and STL-10, and show competitive or superior classification performance when compared to the state-of-the-art. Furthermore, our method achieves state-of-the-art performance on phone classification tasks with the TIMIT dataset, which demonstrates wide applicability of our proposed algorithms to other domains.