Absorbing Discrete Diffusion for Speech Enhancement
For researchers in speech enhancement, this work offers a novel diffusion-based approach that leverages neural audio codecs and non-autoregressive sampling, though performance is only competitive (not SOTA) on standard metrics.
The paper introduces ADDSE, an absorbing discrete diffusion model for speech enhancement that conditions on noisy speech codes to generate clean speech codes, achieving competitive non-intrusive objective metrics, particularly at low SNRs and with few sampling steps.
Inspired by recent developments in neural speech coding and diffusion-based language modeling, we tackle speech enhancement by modeling the conditional distribution of clean speech codes given noisy speech codes using absorbing discrete diffusion. The proposed approach, which we call ADDSE, leverages both the expressive latent space of neural audio codecs and the non-autoregressive sampling procedure of diffusion models. To efficiently model the hierarchical structure of residual vector quantization codes, we propose RQDiT, which combines techniques from RQ-Transformer and diffusion Transformers for non-autoregressive modeling. Results show competitive performance in terms of non-intrusive objective metrics on two datasets, especially at low signal-to-noise ratios and with few sampling steps. Code and audio examples are available online.