Population Based Training for Data Augmentation and Regularization in Speech Recognition
This work addresses the experimental burden and computational cost of finding optimal hyperparameter schedules for speech recognition, offering a simplified approach with incremental improvements.
The paper tackled the problem of optimizing data augmentation and regularization schedules in speech recognition by using population based training to continuously search hyperparameters, resulting in an 8% relative WER improvement and achieving a 5.18% word error rate on LibriSpeech's test-other.
Varying data augmentation policies and regularization over the course of optimization has led to performance improvements over using fixed values. We show that population based training is a useful tool to continuously search those hyperparameters, within a fixed budget. This greatly simplifies the experimental burden and computational cost of finding such optimal schedules. We experiment in speech recognition by optimizing SpecAugment this way, as well as dropout. It compares favorably to a baseline that does not change those hyperparameters over the course of training, with an 8% relative WER improvement. We obtain 5.18% word error rate on LibriSpeech's test-other.