A Geometric Framework for Convolutional Neural Networks
This work provides a foundational geometric approach for gradient descent in neural networks, potentially benefiting researchers in machine learning by enabling new network designs, but it appears incremental as it builds on existing optimization methods without claiming specific performance gains.
The authors tackled the problem of performing gradient descent in neural networks by proposing a geometric framework that uses the inner product space structure of parameters for coordinate-free optimization, and they demonstrated its application to convolutional neural networks with gradients calculated for standard and higher-order loss functions.
In this paper, a geometric framework for neural networks is proposed. This framework uses the inner product space structure underlying the parameter set to perform gradient descent not in a component-based form, but in a coordinate-free manner. Convolutional neural networks are described in this framework in a compact form, with the gradients of standard --- and higher-order --- loss functions calculated for each layer of the network. This approach can be applied to other network structures and provides a basis on which to create new networks.