Analytic Learning of Convolutional Neural Network For Pattern Recognition
This work addresses the training efficiency problem for researchers and practitioners using CNNs, offering an incremental improvement over existing analytic methods by extending them from MLPs to CNNs.
The authors tackled the problem of slow and resource-intensive training of convolutional neural networks (CNNs) by proposing an analytic learning method (ACnnL) that obtains weights in one epoch, achieving significantly faster training with reasonably close prediction accuracies to back-propagation and showing a unique advantage in small-sample scenarios.
Training convolutional neural networks (CNNs) with back-propagation (BP) is time-consuming and resource-intensive particularly in view of the need to visit the dataset multiple times. In contrast, analytic learning attempts to obtain the weights in one epoch. However, existing attempts to analytic learning considered only the multilayer perceptron (MLP). In this article, we propose an analytic convolutional neural network learning (ACnnL). Theoretically we show that ACnnL builds a closed-form solution similar to its MLP counterpart, but differs in their regularization constraints. Consequently, we are able to answer to a certain extent why CNNs usually generalize better than MLPs from the implicit regularization point of view. The ACnnL is validated by conducting classification tasks on several benchmark datasets. It is encouraging that the ACnnL trains CNNs in a significantly fast manner with reasonably close prediction accuracies to those using BP. Moreover, our experiments disclose a unique advantage of ACnnL under the small-sample scenario when training data are scarce or expensive.