Binary Input Layer: Training of CNN models with binary input data
This work addresses efficient CNN deployment on edge devices by enabling binarization of the input layer, which is incremental but offers practical gains for specific applications.
The paper tackles the problem of binarizing the first layer of CNNs for edge devices, which typically causes significant error increases, by introducing a binary input layer (BIL) that learns bit-specific binary weights. Results show that BIL outperforms full-precision first layers by 1.92 percentage points on multimodal datasets like PAMAP2 while using only 2% of chip area.
For the efficient execution of deep convolutional neural networks (CNN) on edge devices, various approaches have been presented which reduce the bit width of the network parameters down to 1 bit. Binarization of the first layer was always excluded, as it leads to a significant error increase. Here, we present the novel concept of binary input layer (BIL), which allows the usage of binary input data by learning bit specific binary weights. The concept is evaluated on three datasets (PAMAP2, SVHN, CIFAR-10). Our results show that this approach is in particular beneficial for multimodal datasets (PAMAP2) where it outperforms networks using full precision weights in the first layer by 1:92 percentage points (pp) while consuming only 2 % of the chip area.