Semi-supervised Development of ASR Systems for Multilingual Code-switched Speech in Under-resourced Languages
This work addresses the challenge of building ASR for multilingual code-switched speech in under-resourced languages, which is incremental as it applies semi-supervised methods to a specific domain.
The paper tackled the problem of developing automatic speech recognition (ASR) systems for under-resourced, code-switched speech in five South African languages by comparing separate bilingual and unified five-lingual approaches with semi-supervised training, finding that batch-wise training improved results and bilingual systems performed better but benefited more from pseudo-labels generated by the unified system.
This paper reports on the semi-supervised development of acoustic and language models for under-resourced, code-switched speech in five South African languages. Two approaches are considered. The first constructs four separate bilingual automatic speech recognisers (ASRs) corresponding to four different language pairs between which speakers switch frequently. The second uses a single, unified, five-lingual ASR system that represents all the languages (English, isiZulu, isiXhosa, Setswana and Sesotho). We evaluate the effectiveness of these two approaches when used to add additional data to our extremely sparse training sets. Results indicate that batch-wise semi-supervised training yields better results than a non-batch-wise approach. Furthermore, while the separate bilingual systems achieved better recognition performance than the unified system, they benefited more from pseudo-labels generated by the five-lingual system than from those generated by the bilingual systems.