Deep Convolutional Neural Networks with Zero-Padding: Feature Extraction and Learning
It addresses the problem of improving feature extraction in neural networks for machine learning practitioners, but it appears incremental as it builds on existing DCNN concepts.
This paper demonstrates that deep convolutional neural networks with zero-padding are superior to deep fully connected networks in feature extraction, showing universal consistency and translation-invariance in learning, with theoretical results verified by numerical experiments.
This paper studies the performance of deep convolutional neural networks (DCNNs) with zero-padding in feature extraction and learning. After verifying the roles of zero-padding in enabling translation-equivalence, and pooling in its translation-invariance driven nature, we show that with similar number of free parameters, any deep fully connected networks (DFCNs) can be represented by DCNNs with zero-padding. This demonstrates that DCNNs with zero-padding is essentially better than DFCNs in feature extraction. Consequently, we derive universal consistency of DCNNs with zero-padding and show its translation-invariance in the learning process. All our theoretical results are verified by numerical experiments including both toy simulations and real-data running.