DCFNet: Deep Neural Network with Decomposed Convolutional Filters
This work addresses efficiency and regularization in CNNs for computer vision applications, but it is incremental as it builds on existing decomposition methods.
The paper tackles the problem of reducing trainable parameters and computation in Convolutional Neural Networks (CNNs) by decomposing convolutional filters using pre-fixed bases, achieving maintained accuracy in image classification tasks with significant parameter reduction.
Filters in a Convolutional Neural Network (CNN) contain model parameters learned from enormous amounts of data. In this paper, we suggest to decompose convolutional filters in CNN as a truncated expansion with pre-fixed bases, namely the Decomposed Convolutional Filters network (DCFNet), where the expansion coefficients remain learned from data. Such a structure not only reduces the number of trainable parameters and computation, but also imposes filter regularity by bases truncation. Through extensive experiments, we consistently observe that DCFNet maintains accuracy for image classification tasks with a significant reduction of model parameters, particularly with Fourier-Bessel (FB) bases, and even with random bases. Theoretically, we analyze the representation stability of DCFNet with respect to input variations, and prove representation stability under generic assumptions on the expansion coefficients. The analysis is consistent with the empirical observations.