Approximation Properties of Deep ReLU CNNs
This provides theoretical foundations for deep CNNs, which is incremental as it builds on existing decomposition and connection methods.
The paper tackles the problem of establishing L^2 approximation properties for deep ReLU convolutional neural networks in 2D space, resulting in a universal approximation theorem and extending these properties to networks with ResNet, pre-act ResNet, and MgNet architectures.
This paper focuses on establishing $L^2$ approximation properties for deep ReLU convolutional neural networks (CNNs) in two-dimensional space. The analysis is based on a decomposition theorem for convolutional kernels with a large spatial size and multi-channels. Given the decomposition result, the property of the ReLU activation function, and a specific structure for channels, a universal approximation theorem of deep ReLU CNNs with classic structure is obtained by showing its connection with one-hidden-layer ReLU neural networks (NNs). Furthermore, approximation properties are obtained for one version of neural networks with ResNet, pre-act ResNet, and MgNet architecture based on connections between these networks.