Geometry of Deep Convolutional Networks
This work addresses the challenge of interpreting the internal workings of deep convolutional networks for researchers in machine learning and computer vision, offering a theoretical framework to analyze their geometric properties.
The paper tackles the problem of understanding the geometric transformations in deep convolutional networks by developing a formal procedure to compute preimages of network outputs using dual bases and hyperplane arrangements. The result demonstrates how specific networks flatten nonlinear input manifolds to affine output manifolds, providing insights into classification properties.
We give a formal procedure for computing preimages of convolutional network outputs using the dual basis defined from the set of hyperplanes associated with the layers of the network. We point out the special symmetry associated with arrangements of hyperplanes of convolutional networks that take the form of regular multidimensional polyhedral cones. We discuss the efficiency of large number of layers of nested cones that result from incremental small size convolutions in order to give a good compromise between efficient contraction of data to low dimensions and shaping of preimage manifolds. We demonstrate how a specific network flattens a non linear input manifold to an affine output manifold and discuss its relevance to understanding classification properties of deep networks.