Adversarial training of Keyword Spotting to Minimize TTS Data Overfitting
This addresses the cost and time challenges in keyword spotting development for diverse populations by enabling more effective use of synthetic data, though it is an incremental improvement over existing adversarial methods.
The paper tackles the problem of keyword spotting models overfitting to text-to-speech synthesized data, which degrades accuracy on real speech, by proposing adversarial training to prevent learning TTS-specific features, resulting in up to 12% improvement in accuracy on real speech data.
The keyword spotting (KWS) problem requires large amounts of real speech training data to achieve high accuracy across diverse populations. Utilizing large amounts of text-to-speech (TTS) synthesized data can reduce the cost and time associated with KWS development. However, TTS data may contain artifacts not present in real speech, which the KWS model can exploit (overfit), leading to degraded accuracy on real speech. To address this issue, we propose applying an adversarial training method to prevent the KWS model from learning TTS-specific features when trained on large amounts of TTS data. Experimental results demonstrate that KWS model accuracy on real speech data can be improved by up to 12% when adversarial loss is used in addition to the original KWS loss. Surprisingly, we also observed that the adversarial setup improves accuracy by up to 8%, even when trained solely on TTS and real negative speech data, without any real positive examples.