Segmental Contrastive Predictive Coding for Unsupervised Word Segmentation
This work addresses the challenge of automatic unit detection in speech processing for low-resource scenarios, offering a unified approach that improves over prior methods.
The paper tackles the problem of unsupervised word and phoneme segmentation in zero-resource speech processing by introducing a segmental contrastive predictive coding (SCPC) framework that models signal structure at higher levels like phonemes, and it outperforms existing methods on TIMIT and Buckeye datasets.
Automatic detection of phoneme or word-like units is one of the core objectives in zero-resource speech processing. Recent attempts employ self-supervised training methods, such as contrastive predictive coding (CPC), where the next frame is predicted given past context. However, CPC only looks at the audio signal's frame-level structure. We overcome this limitation with a segmental contrastive predictive coding (SCPC) framework that can model the signal structure at a higher level e.g. at the phoneme level. In this framework, a convolutional neural network learns frame-level representation from the raw waveform via noise-contrastive estimation (NCE). A differentiable boundary detector finds variable-length segments, which are then used to optimize a segment encoder via NCE to learn segment representations. The differentiable boundary detector allows us to train frame-level and segment-level encoders jointly. Typically, phoneme and word segmentation are treated as separate tasks. We unify them and experimentally show that our single model outperforms existing phoneme and word segmentation methods on TIMIT and Buckeye datasets. We analyze the impact of boundary threshold and when is the right time to include the segmental loss in the learning process.