Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Study
This work addresses computational inefficiencies in speech processing for researchers and practitioners, but it is incremental as it builds on existing methods for compression.
The study tackled inefficiencies in speech processing by using discrete speech units to compress data, achieving reasonably good results on 12 speech recognition, 3 translation, and 1 understanding corpora while significantly reducing training time.
Speech signals, typically sampled at rates in the tens of thousands per second, contain redundancies, evoking inefficiencies in sequence modeling. High-dimensional speech features such as spectrograms are often used as the input for the subsequent model. However, they can still be redundant. Recent investigations proposed the use of discrete speech units derived from self-supervised learning representations, which significantly compresses the size of speech data. Applying various methods, such as de-duplication and subword modeling, can further compress the speech sequence length. Hence, training time is significantly reduced while retaining notable performance. In this study, we undertake a comprehensive and systematic exploration into the application of discrete units within end-to-end speech processing models. Experiments on 12 automatic speech recognition, 3 speech translation, and 1 spoken language understanding corpora demonstrate that discrete units achieve reasonably good results in almost all the settings. We intend to release our configurations and trained models to foster future research efforts.