Thank you for Attention: A survey on Attention-based Artificial Neural Networks for Automatic Speech Recognition
This is an incremental survey paper that synthesizes existing research on attention mechanisms for the automatic speech recognition community.
The paper provides a comprehensive review of attention models used in automatic speech recognition systems, focusing on their development and evolution in recurrent neural network- and Transformer-based architectures for offline and streaming tasks.
Attention is a very popular and effective mechanism in artificial neural network-based sequence-to-sequence models. In this survey paper, a comprehensive review of the different attention models used in developing automatic speech recognition systems is provided. The paper focuses on the development and evolution of attention models for offline and streaming speech recognition within recurrent neural network- and Transformer- based architectures.