A Comparative Study on E-Branchformer vs Conformer in Speech Recognition, Translation, and Understanding Tasks
This incremental study provides a comprehensive comparison for the speech processing community, potentially guiding architecture choices in practical applications.
This paper compared E-Branchformer and Conformer encoders across speech recognition, translation, and understanding tasks, finding that E-Branchformer achieved comparable or better performance in 20 benchmarks and was more stable during training.
Conformer, a convolution-augmented Transformer variant, has become the de facto encoder architecture for speech processing due to its superior performance in various tasks, including automatic speech recognition (ASR), speech translation (ST) and spoken language understanding (SLU). Recently, a new encoder called E-Branchformer has outperformed Conformer in the LibriSpeech ASR benchmark, making it promising for more general speech applications. This work compares E-Branchformer and Conformer through extensive experiments using different types of end-to-end sequence-to-sequence models. Results demonstrate that E-Branchformer achieves comparable or better performance than Conformer in almost all evaluation sets across 15 ASR, 2 ST, and 3 SLU benchmarks, while being more stable during training. We will release our training configurations and pre-trained models for reproducibility, which can benefit the speech community.