Composite Binary Decomposition Networks
This work addresses efficiency and accuracy issues in binary neural networks for applications like image classification, object detection, and semantic segmentation, representing an incremental improvement over existing methods.
The paper tackles the problem of binary neural networks suffering from long training times and accuracy drops compared to full-precision networks by proposing Composite Binary Decomposition Networks (CBDNet), which composes real-valued tensors with binary tensors and decomposes them to reduce parameters and operations, achieving minor accuracy drops with approximate bit-widths ranging from 4.38 to 5.72 bits across various networks.
Binary neural networks have great resource and computing efficiency, while suffer from long training procedure and non-negligible accuracy drops, when comparing to the full-precision counterparts. In this paper, we propose the composite binary decomposition networks (CBDNet), which first compose real-valued tensor of each layer with a limited number of binary tensors, and then decompose some conditioned binary tensors into two low-rank binary tensors, so that the number of parameters and operations are greatly reduced comparing to the original ones. Experiments demonstrate the effectiveness of the proposed method, as CBDNet can approximate image classification network ResNet-18 using 5.25 bits, VGG-16 using 5.47 bits, DenseNet-121 using 5.72 bits, object detection networks SSD300 using 4.38 bits, and semantic segmentation networks SegNet using 5.18 bits, all with minor accuracy drops.