A PCA-Based Convolutional Network
This addresses the problem of computational inefficiency in deep learning for researchers and practitioners, though it is incremental as it builds on existing unsupervised and PCA methods.
The paper tackles unsupervised deep learning by proposing a PCA-based Convolutional Network (PCN) that uses PCA for filter learning and binary hashing, achieving competitive or better performance than state-of-the-art models on tasks like handwritten digits recognition, face recognition, and texture classification, with significantly improved efficiency due to no backpropagation.
In this paper, we propose a novel unsupervised deep learning model, called PCA-based Convolutional Network (PCN). The architecture of PCN is composed of several feature extraction stages and a nonlinear output stage. Particularly, each feature extraction stage includes two layers: a convolutional layer and a feature pooling layer. In the convolutional layer, the filter banks are simply learned by PCA. In the nonlinear output stage, binary hashing is applied. For the higher convolutional layers, the filter banks are learned from the feature maps that were obtained in the previous stage. To test PCN, we conducted extensive experiments on some challenging tasks, including handwritten digits recognition, face recognition and texture classification. The results show that PCN performs competitive with or even better than state-of-the-art deep learning models. More importantly, since there is no back propagation for supervised finetuning, PCN is much more efficient than existing deep networks.