Analytic Convolutional Layer: A Step to Analytic Neural Network
This work addresses the need for more interpretable and efficient neural networks in computer vision, though it appears incremental as it builds on existing kernel-based methods.
The paper tackles the problem of embedding prior knowledge in convolutional layers by introducing the Analytic Convolutional Layer (ACL), which combines analytical and traditional kernels, resulting in improved feature representation with fewer parameters and increased reliability.
The prevailing approach to embedding prior knowledge within convolutional layers typically includes the design of steerable kernels or their modulation using designated kernel banks. In this study, we introduce the Analytic Convolutional Layer (ACL), an innovative model-driven convolutional layer, which is a mosaic of analytical convolution kernels (ACKs) and traditional convolution kernels. ACKs are characterized by mathematical functions governed by analytic kernel parameters (AKPs) learned in training process. Learnable AKPs permit the adaptive update of incorporated knowledge to align with the features representation of data. Our extensive experiments demonstrate that the ACLs not only have a remarkable capacity for feature representation with a reduced number of parameters but also attain increased reliability through the analytical formulation of ACKs. Furthermore, ACLs offer a means for neural network interpretation, thereby paving the way for the intrinsic interpretability of neural network. The source code will be published in company with the paper.