Cross-lingual topic prediction for speech using translations
This could aid humanitarian applications like crisis response by quickly assessing foreign low-resource language speech, though it is incremental as it builds on existing translation models.
The paper tackles the problem of classifying speech utterances by topic in a low-resource language using only a small amount of translated speech data, achieving over 70% accuracy on 1-minute segments, which is a 20% improvement over a baseline.
Given a large amount of unannotated speech in a low-resource language, can we classify the speech utterances by topic? We consider this question in the setting where a small amount of speech in the low-resource language is paired with text translations in a high-resource language. We develop an effective cross-lingual topic classifier by training on just 20 hours of translated speech, using a recent model for direct speech-to-text translation. While the translations are poor, they are still good enough to correctly classify the topic of 1-minute speech segments over 70% of the time - a 20% improvement over a majority-class baseline. Such a system could be useful for humanitarian applications like crisis response, where incoming speech in a foreign low-resource language must be quickly assessed for further action.