HiFi-VC: High Quality ASR-Based Voice Conversion
This work addresses the need for high-quality any-to-any voice conversion in applications like entertainment and speech data generation, representing an incremental improvement over existing methods.
The paper tackled the problem of improving any-to-any voice conversion quality, which was inferior to natural speech, by proposing a new pipeline using ASR features, pitch tracking, and a waveform prediction model, resulting in outperforming modern baselines in voice quality, similarity, and consistency.
The goal of voice conversion (VC) is to convert input voice to match the target speaker's voice while keeping text and prosody intact. VC is usually used in entertainment and speaking-aid systems, as well as applied for speech data generation and augmentation. The development of any-to-any VC systems, which are capable of generating voices unseen during model training, is of particular interest to both researchers and the industry. Despite recent progress, any-to-any conversion quality is still inferior to natural speech. In this work, we propose a new any-to-any voice conversion pipeline. Our approach uses automated speech recognition (ASR) features, pitch tracking, and a state-of-the-art waveform prediction model. According to multiple subjective and objective evaluations, our method outperforms modern baselines in terms of voice quality, similarity and consistency.