Harmonic Decompositions of Convolutional Networks
This work provides a theoretical foundation for understanding convolutional networks, which is incremental as it builds on existing functional analysis tools to derive new insights.
The authors tackled the problem of characterizing the function space and smoothness of convolutional networks using reproducing kernel Hilbert spaces, resulting in a harmonic decomposition analogous to spherical harmonics and statistical bounds that reveal a trade-off between approximation and estimation errors.
We present a description of the function space and the smoothness class associated with a convolutional network using the machinery of reproducing kernel Hilbert spaces. We show that the mapping associated with a convolutional network expands into a sum involving elementary functions akin to spherical harmonics. This functional decomposition can be related to the functional ANOVA decomposition in nonparametric statistics. Building off our functional characterization of convolutional networks, we obtain statistical bounds highlighting an interesting trade-off between the approximation error and the estimation error.