Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators
This work addresses the challenge of deploying energy-efficient AIMC accelerators for diverse deep learning inference tasks, though it is incremental as it builds on prior retraining methods with a more realistic hardware model.
The paper tackled the problem of accuracy loss in deep neural networks (DNNs) when using analog in-memory computing (AIMC) accelerators due to hardware nonidealities, and showed that hardware-aware training can achieve iso-accuracy for various DNN topologies, including CNNs, RNNs, and transformers, on large-scale AI workloads.
Analog in-memory computing (AIMC) -- a promising approach for energy-efficient acceleration of deep learning workloads -- computes matrix-vector multiplications (MVMs) but only approximately, due to nonidealities that often are non-deterministic or nonlinear. This can adversely impact the achievable deep neural network (DNN) inference accuracy as compared to a conventional floating point (FP) implementation. While retraining has previously been suggested to improve robustness, prior work has explored only a few DNN topologies, using disparate and overly simplified AIMC hardware models. Here, we use hardware-aware (HWA) training to systematically examine the accuracy of AIMC for multiple common artificial intelligence (AI) workloads across multiple DNN topologies, and investigate sensitivity and robustness to a broad set of nonidealities. By introducing a new and highly realistic AIMC crossbar-model, we improve significantly on earlier retraining approaches. We show that many large-scale DNNs of various topologies, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and transformers, can in fact be successfully retrained to show iso-accuracy on AIMC. Our results further suggest that AIMC nonidealities that add noise to the inputs or outputs, not the weights, have the largest impact on DNN accuracy, and that RNNs are particularly robust to all nonidealities.