Takashi Shinozaki

Takashi Shinozaki

This study proposes a novel biologically-motivated learning method for deep convolutional neural networks (CNNs). The combination of CNNs and back propagation (BP) learning is the most powerful method in recent machine learning regimes. However, it requires large labeled data for training, and this requirement can occasionally become a barrier for real world applications. To address this problem and utilize unlabeled data, I propose to introduce unsupervised competitive learning which only requires forward propagating signals as a pre-training method for CNNs. The method was evaluated by image discrimination tasks using MNIST, CIFAR-10, and ImageNet datasets, and it achieved a state-of-the-art performance as a biologically-motivated method in the ImageNet experiment. The results suggested that the method enables higher-level learning representations solely from forward propagating signals without a backward error signal for the learning of convolutional layers. The proposed method could be useful for a variety of poorly labeled data, for example, time series or medical data.

Takashi Shinozaki

In this study, we propose the integration of competitive learning into convolutional neural networks (CNNs) to improve the representation learning and efficiency of fine-tuning. Conventional CNNs use back propagation learning, and it enables powerful representation learning by a discrimination task. However, it requires huge amount of labeled data, and acquisition of labeled data is much harder than that of unlabeled data. Thus, efficient use of unlabeled data is getting crucial for DNNs. To address the problem, we introduce unsupervised competitive learning into the convolutional layer, and utilize unlabeled data for effective representation learning. The results of validation experiments using a toy model demonstrated that strong representation learning effectively extracted bases of images into convolutional filters using unlabeled data, and accelerated the speed of the fine-tuning of subsequent supervised back propagation learning. The leverage was more apparent when the number of filters was sufficiently large, and, in such a case, the error rate steeply decreased in the initial phase of fine-tuning. Thus, the proposed method enlarged the number of filters in CNNs, and enabled a more detailed and generalized representation. It could provide a possibility of not only deep but broad neural networks.

Takashi Shinozaki

In this study, we propose a novel deep neural network and its supervised learning method that uses a feedforward supervisory signal. The method is inspired by the human visual system and performs human-like association-based learning without any backward error propagation. The feedforward supervisory signal that produces the correct result is preceded by the target signal and associates its confirmed label with the classification result of the target signal. It effectively uses a large amount of information from the feedforward signal, and forms a continuous and rich learning representation. The method is validated using visual recognition tasks on the MNIST handwritten dataset.

Takashi Shinozaki, Yasushi Naruse

We propose a novel learning method for multilayered neural networks which uses feedforward supervisory signal and associates classification of a new input with that of pre-trained input. The proposed method effectively uses rich input information in the earlier layer for robust leaning and revising internal representation in a multilayer neural network.