Non-Volatile Memory Array Based Quantization- and Noise-Resilient LSTM Neural Networks
This work addresses the need for low-power accelerators for LSTM models in edge computing, offering a domain-specific improvement that is incremental in method but delivers strong efficiency gains.
The paper tackles the problem of power-efficient LSTM inference at the edge by applying quantization-aware training to LSTM models, showing that 4-bit NVM weights and ADCs achieve performance equivalent to floating-point baselines while tolerating quantization and analog noise, with benchmarks indicating at least 2.4x better computing efficiency and 40x higher area efficiency than traditional digital approaches.
In cloud and edge computing models, it is important that compute devices at the edge be as power efficient as possible. Long short-term memory (LSTM) neural networks have been widely used for natural language processing, time series prediction and many other sequential data tasks. Thus, for these applications there is increasing need for low-power accelerators for LSTM model inference at the edge. In order to reduce power dissipation due to data transfers within inference devices, there has been significant interest in accelerating vector-matrix multiplication (VMM) operations using non-volatile memory (NVM) weight arrays. In NVM array-based hardware, reduced bit-widths also significantly increases the power efficiency. In this paper, we focus on the application of quantization-aware training algorithm to LSTM models, and the benefits these models bring in terms of resilience against both quantization error and analog device noise. We have shown that only 4-bit NVM weights and 4-bit ADC/DACs are needed to produce equivalent LSTM network performance as floating-point baseline. Reasonable levels of ADC quantization noise and weight noise can be naturally tolerated within our NVMbased quantized LSTM network. Benchmark analysis of our proposed LSTM accelerator for inference has shown at least 2.4x better computing efficiency and 40x higher area efficiency than traditional digital approaches (GPU, FPGA, and ASIC). Some other novel approaches based on NVM promise to deliver higher computing efficiency (up to 4.7x) but require larger arrays with potential higher error rates.