Factorised Speaker-environment Adaptive Training of Conformer Speech Recognition Systems
This addresses the problem of robust speech recognition in noisy and diverse conditions for ASR systems, offering incremental improvements through factorised adaptation.
The paper tackled the challenge of variability in natural speech by proposing a Bayesian factorised speaker-environment adaptive training approach for Conformer ASR models, resulting in up to 3.1% absolute (10.4% relative) word error rate reductions compared to baselines.
Rich sources of variability in natural speech present significant challenges to current data intensive speech recognition technologies. To model both speaker and environment level diversity, this paper proposes a novel Bayesian factorised speaker-environment adaptive training and test time adaptation approach for Conformer ASR models. Speaker and environment level characteristics are separately modeled using compact hidden output transforms, which are then linearly or hierarchically combined to represent any speaker-environment combination. Bayesian learning is further utilized to model the adaptation parameter uncertainty. Experiments on the 300-hr WHAM noise corrupted Switchboard data suggest that factorised adaptation consistently outperforms the baseline and speaker label only adapted Conformers by up to 3.1% absolute (10.4% relative) word error rate reductions. Further analysis shows the proposed method offers potential for rapid adaption to unseen speaker-environment conditions.