Improving Autoregressive NLP Tasks via Modular Linearized Attention
This work addresses efficiency challenges for NLP models deployed at the edge, though it appears incremental as it builds on existing attention mechanisms.
The paper tackled the problem of improving computational efficiency for autoregressive NLP tasks in resource-constrained environments by proposing modular linearized attention (MLA), which achieved notable speedups and competitive performance on tasks like speech-to-text translation and text-to-spectrogram.
Various natural language processing (NLP) tasks necessitate models that are efficient and small based on their ultimate application at the edge or in other resource-constrained environments. While prior research has reduced the size of these models, increasing computational efficiency without considerable performance impacts remains difficult, especially for autoregressive tasks. This paper proposes modular linearized attention (MLA), which combines multiple efficient attention mechanisms, including cosFormer, to maximize inference quality while achieving notable speedups. We validate this approach on several autoregressive NLP tasks, including speech-to-text neural machine translation (S2T NMT), speech-to-text simultaneous translation (SimulST), and autoregressive text-to-spectrogram, noting efficiency gains on TTS and competitive performance for NMT and SimulST during training and inference.