Axial Residual Networks for CycleGAN-based Voice Conversion
This addresses voice conversion for applications like speech synthesis, but it is incremental as it builds on existing CycleGAN methods with architectural and loss modifications.
The paper tackles non-parallel voice conversion by proposing a CycleGAN-based model that transforms high-resolution spectrograms directly, preserving speech content while changing speaker identity, and it outperforms Scyclone and matches or exceeds CycleGAN-VC2 without a neural vocoder.
We propose a novel architecture and improved training objectives for non-parallel voice conversion. Our proposed CycleGAN-based model performs a shape-preserving transformation directly on a high frequency-resolution magnitude spectrogram, converting its style (i.e. speaker identity) while preserving the speech content. Throughout the entire conversion process, the model does not resort to compressed intermediate representations of any sort (e.g. mel spectrogram, low resolution spectrogram, decomposed network feature). We propose an efficient axial residual block architecture to support this expensive procedure and various modifications to the CycleGAN losses to stabilize the training process. We demonstrate via experiments that our proposed model outperforms Scyclone and shows a comparable or better performance to that of CycleGAN-VC2 even without employing a neural vocoder.