Comparative Analysis of Polynomial and Rational Approximations of Hyperbolic Tangent Function for VLSI Implementation
This work addresses the need for efficient VLSI implementations of activation functions to optimize performance and area in neural network accelerators, but it is incremental as it focuses on comparative analysis rather than introducing new methods.
The paper tackles the lack of comparative studies on hardware implementations of the hyperbolic tangent activation function for neural network accelerators, presenting an analysis of polynomial and rational approximation methods.
Deep neural networks yield the state-of-the-art results in many computer vision and human machine interface applications such as object detection, speech recognition etc. Since, these networks are computationally expensive, customized accelerators are designed for achieving the required performance at lower cost and power. One of the key building blocks of these neural networks is non-linear activation function such as sigmoid, hyperbolic tangent (tanh), and ReLU. A low complexity accurate hardware implementation of the activation function is required to meet the performance and area targets of the neural network accelerators. Even though, various methods and implementations of tanh activation function have been published, a comparative study is missing. This paper presents comparative analysis of polynomial and rational methods and their hardware implementation.