Improving Textless Spoken Language Understanding with Discrete Units as Intermediate Target
This work addresses SLU for unwritten languages where paired transcripts are unavailable, offering an incremental improvement over existing textless methods.
The paper tackled the problem of textless spoken language understanding (SLU) by using discrete units as intermediate guidance to improve performance, surpassing baseline methods on five benchmark corpora and enhancing few-shot learning and noise handling.
Spoken Language Understanding (SLU) is a task that aims to extract semantic information from spoken utterances. Previous research has made progress in end-to-end SLU by using paired speech-text data, such as pre-trained Automatic Speech Recognition (ASR) models or paired text as intermediate targets. However, acquiring paired transcripts is expensive and impractical for unwritten languages. On the other hand, Textless SLU extracts semantic information from speech without utilizing paired transcripts. However, the absence of intermediate targets and training guidance for textless SLU often results in suboptimal performance. In this work, inspired by the content-disentangled discrete units from self-supervised speech models, we proposed to use discrete units as intermediate guidance to improve textless SLU performance. Our method surpasses the baseline method on five SLU benchmark corpora. Additionally, we find that unit guidance facilitates few-shot learning and enhances the model's ability to handle noise.