Hadamard Domain Training with Integers for Class Incremental Quantized Learning
This work addresses the challenge of high compute and memory demands for continual learning in privacy-sensitive and low-latency edge applications, offering a practical solution with incremental improvements in quantization efficiency.
The paper tackles the problem of enabling efficient continual learning on resource-constrained edge platforms by proposing a technique using Hadamard transforms and integer matrix multiplications to reduce computational precision without significant accuracy loss. It achieves less than 0.5% and 3% accuracy degradation when quantizing all matrix multiplications to 4-bits with 8-bit accumulators on human activity recognition datasets and CIFAR100.
Continual learning is a desirable feature in many modern machine learning applications, which allows in-field adaptation and updating, ranging from accommodating distribution shift, to fine-tuning, and to learning new tasks. For applications with privacy and low latency requirements, the compute and memory demands imposed by continual learning can be cost-prohibitive for resource-constraint edge platforms. Reducing computational precision through fully quantized training (FQT) simultaneously reduces memory footprint and increases compute efficiency for both training and inference. However, aggressive quantization especially integer FQT typically degrades model accuracy to unacceptable levels. In this paper, we propose a technique that leverages inexpensive Hadamard transforms to enable low-precision training with only integer matrix multiplications. We further determine which tensors need stochastic rounding and propose tiled matrix multiplication to enable low-bit width accumulators. We demonstrate the effectiveness of our technique on several human activity recognition datasets and CIFAR100 in a class incremental learning setting. We achieve less than 0.5% and 3% accuracy degradation while we quantize all matrix multiplications inputs down to 4-bits with 8-bit accumulators.