Structured Convolution Matrices for Energy-efficient Deep learning
This work addresses energy efficiency in deep learning for applications like embedded systems, though it appears incremental as it builds on existing connections between neuromorphic architectures and structured matrices.
The authors tackled the problem of energy-efficient deep learning by developing deep convolutional networks using structured convolutional matrices, achieving state-of-the-art trade-offs between energy efficiency and classification accuracy for image recognition tasks, with concrete improvements in energy usage and accuracy metrics.
We derive a relationship between network representation in energy-efficient neuromorphic architectures and block Toplitz convolutional matrices. Inspired by this connection, we develop deep convolutional networks using a family of structured convolutional matrices and achieve state-of-the-art trade-off between energy efficiency and classification accuracy for well-known image recognition tasks. We also put forward a novel method to train binary convolutional networks by utilising an existing connection between noisy-rectified linear units and binary activations.