Convolutional Neural Networks Demystified: A Matched Filtering Perspective Based Tutorial
This is an incremental tutorial aimed at researchers and practitioners in machine learning to improve understanding and development of CNNs.
The paper tackles the lack of physical intuition in convolutional neural networks (CNNs) by reinterpreting their convolution operation as a matched filter, providing a step-by-step tutorial to demystify their principles and operation.
Deep Neural Networks (DNN) and especially Convolutional Neural Networks (CNN) are a de-facto standard for the analysis of large volumes of signals and images. Yet, their development and underlying principles have been largely performed in an ad-hoc and black box fashion. To help demystify CNNs, we revisit their operation from first principles and a matched filtering perspective. We establish that the convolution operation within CNNs, their very backbone, represents a matched filter which examines the input signal/image for the presence of pre-defined features. This perspective is shown to be physically meaningful, and serves as a basis for a step-by-step tutorial on the operation of CNNs, including pooling, zero padding, various ways of dimensionality reduction. Starting from first principles, both the feed-forward pass and the learning stage (via back-propagation) are illuminated in detail, both through a worked-out numerical example and the corresponding visualizations. It is our hope that this tutorial will help shed new light and physical intuition into the understanding and further development of deep neural networks.