On the Universal Approximation Property of Deep Fully Convolutional Neural Networks
This provides theoretical guarantees for convolutional neural networks in symmetric function approximation, which is incremental but important for understanding their capabilities in domains like image processing.
The paper tackles the problem of approximating shift-invariant or equivariant functions using deep fully convolutional networks, proving that both residual and non-residual variants can achieve universal approximation with constant channel width, while also showing that fewer channels or smaller kernels fail to be universal approximators.
We study the approximation of shift-invariant or equivariant functions by deep fully convolutional networks from the dynamical systems perspective. We prove that deep residual fully convolutional networks and their continuous-layer counterpart can achieve universal approximation of these symmetric functions at constant channel width. Moreover, we show that the same can be achieved by non-residual variants with at least 2 channels in each layer and convolutional kernel size of at least 2. In addition, we show that these requirements are necessary, in the sense that networks with fewer channels or smaller kernels fail to be universal approximators.