Adaptive Precision Training: Quantify Back Propagation in Neural Networks with Fixed-point Numbers
This work addresses the problem of efficient and accurate training for deep learning practitioners, but it appears incremental as it builds on existing quantization techniques.
The paper tackles the challenge of applying quantization to neural network training, which typically causes significant accuracy loss, by proposing an adaptive precision training method using fixed-point numbers for backpropagation.
Adaptive Precision Training: Quantify Back Propagation in Neural Networks with Fixed-point Numbers. Recent emerged quantization technique has been applied to inference of deep neural networks for fast and efficient execution. However, directly applying quantization in training can cause significant accuracy loss, thus remaining an open challenge.