Learning Disentangled Speech Representations
This work addresses a critical gap for researchers in speech processing by providing a benchmark dataset and evaluation framework to support the development of more robust and interpretable methods, but it is incremental as it focuses on dataset creation and evaluation rather than novel methods.
The authors tackled the lack of datasets for evaluating disentangled speech representations by proposing SynSpeech, a synthetic dataset with controlled variations, and found it enables benchmarking with promising disentanglement for simpler features like gender and speaking style, though challenges remain for complex attributes like speaker identity.
Disentangled representation learning in speech processing has lagged behind other domains, largely due to the lack of datasets with annotated generative factors for robust evaluation. To address this, we propose SynSpeech, a novel large-scale synthetic speech dataset specifically designed to enable research on disentangled speech representations. SynSpeech includes controlled variations in speaker identity, spoken text, and speaking style, with three dataset versions to support experimentation at different levels of complexity. In this study, we present a comprehensive framework to evaluate disentangled representation learning techniques, applying both linear probing and established supervised disentanglement metrics to assess the modularity, compactness, and informativeness of the representations learned by a state-of-the-art model. Using the RAVE model as a test case, we find that SynSpeech facilitates benchmarking across a range of factors, achieving promising disentanglement of simpler features like gender and speaking style, while highlighting challenges in isolating complex attributes like speaker identity. This benchmark dataset and evaluation framework fills a critical gap, supporting the development of more robust and interpretable speech representation learning methods.