Learning Audio-Text Agreement for Open-vocabulary Keyword Spotting
This addresses the problem of flexible keyword detection for users in speech recognition systems, though it is incremental as it builds on existing cross-modal approaches.
The paper tackles the problem of open-vocabulary keyword spotting by proposing a method that compares audio queries with text keywords using cross-modal matching, eliminating the need for speech enrollment. It achieves competitive results on various evaluation sets and introduces the LibriPhrase dataset for efficient training.
In this paper, we propose a novel end-to-end user-defined keyword spotting method that utilizes linguistically corresponding patterns between speech and text sequences. Unlike previous approaches requiring speech keyword enrollment, our method compares input queries with an enrolled text keyword sequence. To place the audio and text representations within a common latent space, we adopt an attention-based cross-modal matching approach that is trained in an end-to-end manner with monotonic matching loss and keyword classification loss. We also utilize a de-noising loss for the acoustic embedding network to improve robustness in noisy environments. Additionally, we introduce the LibriPhrase dataset, a new short-phrase dataset based on LibriSpeech for efficiently training keyword spotting models. Our proposed method achieves competitive results on various evaluation sets compared to other single-modal and cross-modal baselines.