Streaming Transformer Transducer Based Speech Recognition Using Non-Causal Convolution
This work addresses streaming speech recognition for applications like dictation and voice assistants, representing an incremental improvement over existing methods.
This paper tackles the problem of streaming speech recognition by improving the transformer transducer with non-causal convolution to leverage lookahead context, achieving relative WERR improvements of 5.1%, 14.5%, and 8.4% on different scenarios compared to a baseline.
This paper improves the streaming transformer transducer for speech recognition by using non-causal convolution. Many works apply the causal convolution to improve streaming transformer ignoring the lookahead context. We propose to use non-causal convolution to process the center block and lookahead context separately. This method leverages the lookahead context in convolution and maintains similar training and decoding efficiency. Given the similar latency, using the non-causal convolution with lookahead context gives better accuracy than causal convolution, especially for open-domain dictation scenarios. Besides, this paper applies talking-head attention and a novel history context compression scheme to further improve the performance. The talking-head attention improves the multi-head self-attention by transferring information among different heads. The history context compression method introduces more extended history context compactly. On our in-house data, the proposed methods improve a small Emformer baseline with lookahead context by relative WERR 5.1\%, 14.5\%, 8.4\% on open-domain dictation, assistant general scenarios, and assistant calling scenarios, respectively.