Design Challenges of Neural Network Acceleration Using Stochastic Computing
This work addresses energy and area efficiency for neural network acceleration in IoT devices, but it is incremental as it compares existing methods without introducing new techniques.
The paper tackles the challenge of deploying neural networks on resource-limited IoT devices by evaluating stochastic computing designs, finding that the BISC architecture outperforms ESL architectures with 50x faster speed, 5.7x and 2.9x less area, and 7.8x and 1.8x less power consumption on the LeNet-5 model for MNIST digit recognition.
The enormous and ever-increasing complexity of state-of-the-art neural networks (NNs) has impeded the deployment of deep learning on resource-limited devices such as the Internet of Things (IoTs). Stochastic computing exploits the inherent amenability to approximation characteristic of NNs to reduce their energy and area footprint, two critical requirements of small embedded devices suitable for the IoTs. This report evaluates and compares two recently proposed stochastic-based NN designs, referred to as BISC (Binary Interfaced Stochastic Computing) by Sim and Lee, 2017, and ESL (Extended Stochastic Logic) by Canals et al., 2016. Using analysis and simulation, we compare three distinct implementations of these designs in terms of performance, power consumption, area, and accuracy. We also discuss the overall challenges faced in adopting stochastic computing for building NNs. We find that BISC outperforms the other architectures when executing the LeNet-5 NN model applied to the MNIST digit recognition dataset. Our analysis and simulation experiments indicate that this architecture is around 50X faster, occupies 5.7X and 2.9X less area, and consumes 7.8X and 1.8X less power than the two ESL architectures.