Is Each Layer Non-trivial in CNN?
This work addresses the efficiency and interpretability of deep CNNs for researchers and practitioners, but it is incremental as it builds on existing ResNet architectures.
The paper investigates whether each layer in convolutional neural networks (CNNs) is non-trivial by replacing convolution kernels with zeros and testing on datasets, finding that similar or identical performance can be achieved, indicating some kernels are trivial and regular in ResNet.
Convolutional neural network (CNN) models have achieved great success in many fields. With the advent of ResNet, networks used in practice are getting deeper and wider. However, is each layer non-trivial in networks? To answer this question, we trained a network on the training set, then we replace the network convolution kernels with zeros and test the result models on the test set. We compared experimental results with baseline and showed that we can reach similar or even the same performances. Although convolution kernels are the cores of networks, we demonstrate that some of them are trivial and regular in ResNet.