Information Retrieval for ZeroSpeech 2021: The Submission by University of Wroclaw
This work addresses the challenge of speech processing without labeled data, but it is incremental as it builds on existing baseline representations.
The authors tackled the problem of low-resource speech processing in the ZeroSpeech 2021 Challenge by refining unsupervised CPC-based representations, showing that simple methods could match or outperform more computationally expensive solutions, with specific improvements in pattern matching and retrieval tasks.
We present a number of low-resource approaches to the tasks of the Zero Resource Speech Challenge 2021. We build on the unsupervised representations of speech proposed by the organizers as a baseline, derived from CPC and clustered with the k-means algorithm. We demonstrate that simple methods of refining those representations can narrow the gap, or even improve upon the solutions which use a high computational budget. The results lead to the conclusion that the CPC-derived representations are still too noisy for training language models, but stable enough for simpler forms of pattern matching and retrieval.