Towards Accurate Binary Convolutional Neural Network
This work addresses the problem of accuracy degradation in binary CNNs, which is significant for researchers and practitioners interested in deploying CNNs on resource-constrained devices.
This paper tackles the problem of accuracy degradation in binary convolutional neural networks (CNNs) by proposing a method that approximates full-precision weights with a linear combination of multiple binary weight bases and uses multiple binary activations. The proposed ABC-Net achieves comparable prediction accuracy to full-precision CNNs on ImageNet and forest trail datasets.
We introduce a novel scheme to train binary convolutional neural networks (CNNs) -- CNNs with weights and activations constrained to {-1,+1} at run-time. It has been known that using binary weights and activations drastically reduce memory size and accesses, and can replace arithmetic operations with more efficient bitwise operations, leading to much faster test-time inference and lower power consumption. However, previous works on binarizing CNNs usually result in severe prediction accuracy degradation. In this paper, we address this issue with two major innovations: (1) approximating full-precision weights with the linear combination of multiple binary weight bases; (2) employing multiple binary activations to alleviate information loss. The implementation of the resulting binary CNN, denoted as ABC-Net, is shown to achieve much closer performance to its full-precision counterpart, and even reach the comparable prediction accuracy on ImageNet and forest trail datasets, given adequate binary weight bases and activations.