Efficient Evaluation of Quantization-Effects in Neural Codecs
This work addresses the challenge of evaluating neural codecs for researchers and practitioners, but it is incremental as it focuses on improving evaluation efficiency rather than introducing a new codec paradigm.
The paper tackles the problem of costly and time-consuming evaluation of quantization effects in neural codecs by proposing an efficient framework using simulated data and low-complexity networks, which reduces training time and computational requirements and leads to a modification to stabilize training with the straight-through estimator.
Neural codecs, comprising an encoder, quantizer, and decoder, enable signal transmission at exceptionally low bitrates. Training these systems requires techniques like the straight-through estimator, soft-to-hard annealing, or statistical quantizer emulation to allow a non-zero gradient across the quantizer. Evaluating the effect of quantization in neural codecs, like the influence of gradient passing techniques on the whole system, is often costly and time-consuming due to training demands and the lack of affordable and reliable metrics. This paper proposes an efficient evaluation framework for neural codecs using simulated data with a defined number of bits and low-complexity neural encoders/decoders to emulate the non-linear behavior in larger networks. Our system is highly efficient in terms of training time and computational and hardware requirements, allowing us to uncover distinct behaviors in neural codecs. We propose a modification to stabilize training with the straight-through estimator based on our findings. We validate our findings against an internal neural audio codec and against the state-of-the-art descript-audio-codec.