Exploring the Benefits of Tokenization of Discrete Acoustic Units
This work addresses a gap in audio-based representations for speech processing, offering incremental improvements to existing discrete language-modeling techniques.
The paper tackles the problem of tokenizing discrete acoustic units (DAUs) and phonemes, showing that this approach yields significant improvements in performance, training speed, and inference speed across three prediction tasks: grapheme-to-phoneme, grapheme-to-DAUs, and unsupervised speech generation.
Tokenization algorithms that merge the units of a base vocabulary into larger, variable-rate units have become standard in natural language processing tasks. This idea, however, has been mostly overlooked when the vocabulary consists of phonemes or Discrete Acoustic Units (DAUs), an audio-based representation that is playing an increasingly important role due to the success of discrete language-modeling techniques. In this paper, we showcase the advantages of tokenization of phonetic units and of DAUs on three prediction tasks: grapheme-to-phoneme, grapheme-to-DAUs, and unsupervised speech generation using DAU language modeling. We demonstrate that tokenization yields significant improvements in terms of performance, as well as training and inference speed, across all three tasks. We also offer theoretical insights to provide some explanation for the superior performance observed.