Resource-Efficient Transformer Architecture: Optimizing Memory and Execution Time for Real-Time Applications
This work addresses resource constraints for real-time applications on edge devices, but it is incremental as it builds on existing transformer optimizations like pruning and quantization.
This paper tackled the problem of high memory and computational demands of transformer models on edge devices by proposing a memory-efficient architecture, achieving a 52% reduction in memory usage and a 33% decrease in execution time with minimal accuracy loss.
This paper describes a memory-efficient transformer model designed to drive a reduction in memory usage and execution time by substantial orders of magnitude without impairing the model's performance near that of the original model. Recently, new architectures of transformers were presented, focused on parameter efficiency and computational optimization; however, such models usually require considerable resources in terms of hardware when deployed in real-world applications on edge devices. This approach addresses this concern by halving embedding size and applying targeted techniques such as parameter pruning and quantization to optimize the memory footprint with minimum sacrifices in terms of accuracy. Experimental results include a 52% reduction in memory usage and a 33% decrease in execution time, resulting in better efficiency than state-of-the-art models. This work compared our model with existing compelling architectures, such as MobileBERT and DistilBERT, and proved its feasibility in the domain of resource-friendly deep learning architectures, mainly for applications in real-time and in resource-constrained applications.