Salient Speech Representations Based on Cloned Networks
This work addresses the need for effective feature extraction in speech processing, though it appears incremental as it builds on existing encoder and generative methods.
The paper tackles the problem of extracting salient features from equivalent signals for generative network conditioning, achieving this by training cloned encoder networks to produce identical features across inputs, with application to speech coding and enhancement using WaveNet.
We define salient features as features that are shared by signals that are defined as being equivalent by a system designer. The definition allows the designer to contribute qualitative information. We aim to find salient features that are useful as conditioning for generative networks. We extract salient features by jointly training a set of clones of an encoder network. Each network clone receives as input a different signal from a set of equivalent signals. The objective function encourages the network clones to map their input into a set of features that is identical across the clones. It additionally encourages feature independence and, optionally, reconstruction of a desired target signal by a decoder. As an application, we train a system that extracts a time-sequence of feature vectors of speech and uses it as a conditioning of a WaveNet generative system, facilitating both coding and enhancement.