Acoustic span embeddings for multilingual query-by-example search
This work provides a faster and more accurate method for query-by-example speech search, particularly beneficial for low-resource languages and multilingual applications, improving accessibility and utility for users of such languages.
This paper generalizes acoustic word embeddings to acoustic span embeddings (ASE) for query-by-example (QbE) speech search. The authors apply ASE to arbitrary-length queries in multiple unseen languages, achieving faster search speeds than DTW-based methods and outperforming previous state-of-the-art results on the QUESST 2015 QbE tasks.
Query-by-example (QbE) speech search is the task of matching spoken queries to utterances within a search collection. In low- or zero-resource settings, QbE search is often addressed with approaches based on dynamic time warping (DTW). Recent work has found that methods based on acoustic word embeddings (AWEs) can improve both performance and search speed. However, prior work on AWE-based QbE has primarily focused on English data and with single-word queries. In this work, we generalize AWE training to spans of words, producing acoustic span embeddings (ASE), and explore the application of ASE to QbE with arbitrary-length queries in multiple unseen languages. We consider the commonly used setting where we have access to labeled data in other languages (in our case, several low-resource languages) distinct from the unseen test languages. We evaluate our approach on the QUESST 2015 QbE tasks, finding that multilingual ASE-based search is much faster than DTW-based search and outperforms the best previously published results on this task.