Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition
This work addresses the need for efficient and accurate speech recognition systems, offering a competitive non-autoregressive approach that trains quickly with a fixed architecture on a single GPU, though it is incremental as it adapts self-attention from NLP to CTC in speech.
The authors tackled the problem of improving end-to-end speech recognition by proposing SAN-CTC, a deep, fully self-attentional network for connectionist temporal classification, which achieved character error rates of 4.7% on WSJ eval92 in 1 day and 2.8% on LibriSpeech test-clean in 1 week, outperforming existing CTC models and most encoder-decoder models.
The success of self-attention in NLP has led to recent applications in end-to-end encoder-decoder architectures for speech recognition. Separately, connectionist temporal classification (CTC) has matured as an alignment-free, non-autoregressive approach to sequence transduction, either by itself or in various multitask and decoding frameworks. We propose SAN-CTC, a deep, fully self-attentional network for CTC, and show it is tractable and competitive for end-to-end speech recognition. SAN-CTC trains quickly and outperforms existing CTC models and most encoder-decoder models, with character error rates (CERs) of 4.7% in 1 day on WSJ eval92 and 2.8% in 1 week on LibriSpeech test-clean, with a fixed architecture and one GPU. Similar improvements hold for WERs after LM decoding. We motivate the architecture for speech, evaluate position and downsampling approaches, and explore how label alphabets (character, phoneme, subword) affect attention heads and performance.