MVQ:Towards Efficient DNN Compression and Acceleration with Masked Vector Quantization
This work addresses efficient DNN compression and acceleration for hardware accelerators, offering improvements in energy efficiency and storage, but it is incremental as it builds on existing VQ and pruning techniques.
The paper tackles the problem of significant accuracy loss in conventional vector quantization (VQ) for DNN compression by proposing MVQ, which better approximates important weights with limited codewords using N:M pruning and masked k-means. Experimental results show MVQ outperforms conventional VQ methods, reduces FLOPs, and boosts energy efficiency by 2.3× while reducing systolic array size by 55% in ASIC evaluation.
Vector quantization(VQ) is a hardware-friendly DNN compression method that can reduce the storage cost and weight-loading datawidth of hardware accelerators. However, conventional VQ techniques lead to significant accuracy loss because the important weights are not well preserved. To tackle this problem, a novel approach called MVQ is proposed, which aims at better approximating important weights with a limited number of codewords. At the algorithm level, our approach removes the less important weights through N:M pruning and then minimizes the vector clustering error between the remaining weights and codewords by the masked k-means algorithm. Only distances between the unpruned weights and the codewords are computed, which are then used to update the codewords. At the architecture level, our accelerator implements vector quantization on an EWS (Enhanced weight stationary) CNN accelerator and proposes a sparse systolic array design to maximize the benefits brought by masked vector quantization.\\ Our algorithm is validated on various models for image classification, object detection, and segmentation tasks. Experimental results demonstrate that MVQ not only outperforms conventional vector quantization methods at comparable compression ratios but also reduces FLOPs. Under ASIC evaluation, our MVQ accelerator boosts energy efficiency by 2.3$\times$ and reduces the size of the systolic array by 55\% when compared with the base EWS accelerator. Compared to the previous sparse accelerators, MVQ achieves 1.73$\times$ higher energy efficiency.