Investigation of Training Label Error Impact on RNN-T
This work addresses the impact of label errors on RNN-T models for ASR, offering incremental insights for improving data pipeline design.
The paper investigates how different types of training label errors affect RNN-T-based automatic speech recognition models, finding that deletion errors are more harmful than substitution and insertion errors. It also shows that while mitigation approaches reduce degradation, they cannot fully eliminate the performance gap compared to error-free training.
In this paper, we propose an approach to quantitatively analyze impacts of different training label errors to RNN-T based ASR models. The result shows deletion errors are more harmful than substitution and insertion label errors in RNN-T training data. We also examined label error impact mitigation approaches on RNN-T and found that, though all the methods mitigate the label-error-caused degradation to some extent, they could not remove the performance gap between the models trained with and without the presence of label errors. Based on the analysis results, we suggest to design data pipelines for RNN-T with higher priority on reducing deletion label errors. We also find that ensuring high-quality training labels remains important, despite of the existence of the label error mitigation approaches.