Finite Scalar Quantization Enables Redundant and Transmission-Robust Neural Audio Compression at Low Bit-rates
This work addresses the need for robust and efficient neural audio codecs in speech processing, particularly for transmission over noisy channels, representing an incremental improvement by applying FSQ to audio compression.
The paper tackled the problem of neural audio compression at low bit-rates by introducing NeuCodec, a codec based on Finite Scalar Quantization (FSQ), which showed that FSQ encodes baked-in redundancy, making it robust to noisy channels, with experiments demonstrating vastly superior bit-level perturbation robustness compared to Residual Vector Quantization (RVQ).
Neural Audio Codecs (NACs) have become increasingly adopted in speech processing tasks due to their excellent rate-distortion performance and compatibility with Large Language Models (LLMs) as discrete feature representations for audio generation. While most existing codecs rely on Residual Vector Quantization (RVQ), Finite Scalar Quantization (FSQ) has recently emerged as a compelling alternative that simplifies training and natively supports single codebooks. We introduce NeuCodec, an FSQ-based NAC, and show that FSQ encodes baked-in redundancy which produces an encoding which is robust when transmitted through noisy channels. First, through an encoder distillation experiment, we show that two different encoders can learn to encode identical audio into vastly different code sequences whilst maintaining comparable reconstruction quality with the same quantizer and decoder. Second, we demonstrate that FSQ has vastly superior bit-level perturbation robustness by comparing the performance of RVQ and FSQ codecs when simulating the transmission of code sequences through a noisy channel.