Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Training
This work addresses computational inefficiency in quantization-aware training for deep learning practitioners, offering an incremental improvement by reducing unnecessary updates.
The paper tackles the inefficiency of updating all quantized weights throughout quantization-aware training by observing that many weights stabilize early, termed the partly scratch-off lottery ticket, and proposes a method to freeze these weights, reducing weight updates by 50%-70% and backward FLOPs by 25%-35% while maintaining or improving performance, e.g., boosting 2-bit MobileNetV2 by 5.05%.
Quantization-aware training (QAT) receives extensive popularity as it well retains the performance of quantized networks. In QAT, the contemporary experience is that all quantized weights are updated for an entire training process. In this paper, this experience is challenged based on an interesting phenomenon we observed. Specifically, a large portion of quantized weights reaches the optimal quantization level after a few training epochs, which we refer to as the partly scratch-off lottery ticket. This straightforward-yet-valuable observation naturally inspires us to zero out gradient calculations of these weights in the remaining training period to avoid meaningless updating. To effectively find the ticket, we develop a heuristic method, dubbed lottery ticket scratcher (LTS), which freezes a weight once the distance between the full-precision one and its quantization level is smaller than a controllable threshold. Surprisingly, the proposed LTS typically eliminates 50%-70% weight updating and 25%-35% FLOPs of the backward pass, while still resulting on par with or even better performance than the compared baseline. For example, compared with the baseline, LTS improves 2-bit MobileNetV2 by 5.05%, eliminating 46% weight updating and 23% FLOPs of the backward pass. Code is at url{https://github.com/zysxmu/LTS}.