AxTrain: Hardware-Oriented Neural Network Training for Approximate Inference
This work addresses energy efficiency challenges in neural network inference for hardware applications, but it is incremental as it builds on existing approximate computing methods.
The paper tackles the problem of suboptimal energy-accuracy trade-offs in neural network inference by proposing AxTrain, a hardware-oriented training framework that actively and passively enhances error tolerance, resulting in improved system energy efficiency across various datasets.
The intrinsic error tolerance of neural network (NN) makes approximate computing a promising technique to improve the energy efficiency of NN inference. Conventional approximate computing focuses on balancing the efficiency-accuracy trade-off for existing pre-trained networks, which can lead to suboptimal solutions. In this paper, we propose AxTrain, a hardware-oriented training framework to facilitate approximate computing for NN inference. Specifically, AxTrain leverages the synergy between two orthogonal methods---one actively searches for a network parameters distribution with high error tolerance, and the other passively learns resilient weights by numerically incorporating the noise distributions of the approximate hardware in the forward pass during the training phase. Experimental results from various datasets with near-threshold computing and approximation multiplication strategies demonstrate AxTrain's ability to obtain resilient neural network parameters and system energy efficiency improvement.