Sensitivity-Aware Finetuning for Accuracy Recovery on Deep Learning Hardware
This addresses efficiency issues in deploying models on analog-digital hardware for practitioners, though it is incremental as it builds on existing noise-injection methods.
The paper tackles the problem of slow noise-injection training for accuracy recovery on deep learning hardware by introducing Sensitivity-Aware Finetuning (SAFT), which identifies and freezes noise-sensitive layers to achieve comparable accuracy with 2x to 8x speed improvements.
Existing methods to recover model accuracy on analog-digital hardware in the presence of quantization and analog noise include noise-injection training. However, it can be slow in practice, incurring high computational costs, even when starting from pretrained models. We introduce the Sensitivity-Aware Finetuning (SAFT) approach that identifies noise sensitive layers in a model, and uses the information to freeze specific layers for noise-injection training. Our results show that SAFT achieves comparable accuracy to noise-injection training and is 2x to 8x faster.