Input Layer Binarization with Bit-Plane Encoding
This work addresses the challenge of making BNNs more efficient for edge devices by enabling full binarization, which is incremental as it builds on existing binarization methods but specifically targets the first layer bottleneck.
The paper tackles the problem of binarizing the first layer in Binary Neural Networks (BNNs), which typically causes significant accuracy loss, by proposing a method using bit-plane encoding and depth-wise convolutions to achieve full binarization without data expansion. The result shows that this technique outperforms state-of-the-art methods in accuracy and reduces BMACs on datasets like CIFAR10, SVHN, and CIFAR100 with models such as VGG and ResNet.
Binary Neural Networks (BNNs) use 1-bit weights and activations to efficiently execute deep convolutional neural networks on edge devices. Nevertheless, the binarization of the first layer is conventionally excluded, as it leads to a large accuracy loss. The few works addressing the first layer binarization, typically increase the number of input channels to enhance data representation; such data expansion raises the amount of operations needed and it is feasible only on systems with enough computational resources. In this work, we present a new method to binarize the first layer using directly the 8-bit representation of input data; we exploit the standard bit-planes encoding to extract features bit-wise (using depth-wise convolutions); after a re-weighting stage, features are fused again. The resulting model is fully binarized and our first layer binarization approach is model independent. The concept is evaluated on three classification datasets (CIFAR10, SVHN and CIFAR100) for different model architectures (VGG and ResNet) and, the proposed technique outperforms state of the art methods both in accuracy and BMACs reduction.