SwitchCodec: A High-Fidelity Nerual Audio Codec With Sparse Quantization
This work addresses the performance degradation of neural audio codecs at low bitrates, offering a high-fidelity solution for efficient audio compression.
SwitchCodec introduces Residual Experts Vector Quantization (REVQ) and a multi-tiered discriminator for neural audio compression, achieving PESQ of 2.87 and ViSQOL of 4.27 at 2.67 kbps, with 13% reduction in mel-spectrogram distance and halved training time via post-training strategy.
Neural audio compression has emerged as a promising technology for efficiently representing speech, music, and general audio. However, existing methods suffer from significant performance degradation at limited bitrates, where the available embedding space is sharply constrained. To address this, we propose a universal high-fidelity neural audio compression algorithm featuring Residual Experts Vector Quantization (REVQ), which substantially expands the embedding space with minimal impact on bandwidth. A gentle load-balancing strategy is introduced to ensure the full utilization of this expanded space. Furthermore, we develop a novel multi-tiered discriminator that periodically stratifies STFT spectra, guiding the generator to focus on critical spectral regions. To support multiple bitrates without quality loss at the lower end, we adopt an efficient post-training strategy. Our proposed model achieves impressive performance, with PESQ and ViSQOL scores of 2.87 and 4.27, respectively, at 2.67 kbps bandwidth. The approach effectively reduces spectral blur, decreasing the distance to the original mel-spectrogram by 13%. Notably, our post-training strategy achieves performance comparable to dedicated fixed-bitrate models while reducing the required training time by half. Extensive ablation studies confirm the superiority of our method over baselines.