Approximate FPGA-based LSTMs under Computation Time Constraints
This work addresses the problem of high-performance LSTM execution for real-time applications, representing an incremental improvement through optimization and hardware integration.
The paper tackles the challenge of deploying computationally demanding LSTM networks under strict computation time constraints for latency-sensitive applications like mobile robots and autonomous vehicles, by introducing an approximate computing scheme and FPGA-based architecture that reduces time by up to 6.5x for the same accuracy and increases accuracy by an average of 25x under the same time constraints.
Recurrent Neural Networks and in particular Long Short-Term Memory (LSTM) networks have demonstrated state-of-the-art accuracy in several emerging Artificial Intelligence tasks. However, the models are becoming increasingly demanding in terms of computational and memory load. Emerging latency-sensitive applications including mobile robots and autonomous vehicles often operate under stringent computation time constraints. In this paper, we address the challenge of deploying computationally demanding LSTMs at a constrained time budget by introducing an approximate computing scheme that combines iterative low-rank compression and pruning, along with a novel FPGA-based LSTM architecture. Combined in an end-to-end framework, the approximation method's parameters are optimised and the architecture is configured to address the problem of high-performance LSTM execution in time-constrained applications. Quantitative evaluation on a real-life image captioning application indicates that the proposed methods required up to 6.5x less time to achieve the same application-level accuracy compared to a baseline method, while achieving an average of 25x higher accuracy under the same computation time constraints.