Task-Specific Audio Coding for Machines: Machine-Learned Latent Features Are Codes for That Machine
This work addresses efficient compression for machine processing in audio tasks, offering a domain-specific solution that is incremental in its approach.
The paper tackles the problem of audio coding for machines by introducing a method that compresses and quantizes intermediate features of trained speech/audio models, achieving ultra-low bitrates (less than 200 bps) with minimal performance loss in downstream tasks like automatic speech recognition and audio classification.
Neural audio codecs, leveraging quantization algorithms, have significantly impacted various speech/audio tasks. While high-fidelity reconstruction is paramount for human perception, audio coding for machines (ACoM) prioritizes efficient compression and downstream task performance, disregarding perceptual nuances. This work introduces an efficient ACoM method that can compress and quantize any chosen intermediate feature representation of an already trained speech/audio downstream model. Our approach employs task-specific loss guidance alongside residual vector quantization (RVQ) losses, providing ultra-low bitrates (i.e., less than 200 bps) with a minimal loss of the downstream model performance. The resulting tokenizer is adaptable to various bitrates and model sizes for flexible deployment. Evaluated on automatic speech recognition and audio classification, our method demonstrates its efficacy and potential for broader task and architectural applicability through appropriate regularization.