Binarized Weight Error Networks With a Transition Regularization Term
This work addresses resource efficiency for neural network deployment, particularly in visual tasks, but it is incremental as it builds on existing binarized weight methods.
The paper tackles the problem of resource-efficient neural networks by proposing a binarized weight network with a transition regularization term, resulting in improved representation capacity and stability for shallow networks while reducing computation load, and it outperforms all counterparts on benchmarks like Cifar10 and ImageNet2012.
This paper proposes a novel binarized weight network (BT) for a resource-efficient neural structure. The proposed model estimates a binary representation of weights by taking into account the approximation error with an additional term. This model increases representation capacity and stability, particularly for shallow networks, while the computation load is theoretically reduced. In addition, a novel regularization term is introduced that is suitable for all threshold-based binary precision networks. This term penalizes the trainable parameters that are far from the thresholds at which binary transitions occur. This step promotes a swift modification for binary-precision responses at train time. The experimental results are carried out for two sets of tasks: visual classification and visual inverse problems. Benchmarks for Cifar10, SVHN, Fashion, ImageNet2012, Set5, Set14, Urban and BSD100 datasets show that our method outperforms all counterparts with binary precision.