Diffusion Buffer: Online Diffusion-based Speech Enhancement with Sub-Second Latency
This work addresses the challenge of high computational cost in diffusion models for real-time speech enhancement, enabling practical online applications with sub-second latency.
The paper tackled the problem of making diffusion models practical for real-time speech enhancement by introducing a sliding window diffusion framework that progressively corrupts speech signals in a buffer, enabling a trade-off between performance and latency. The result was a method that outperforms standard diffusion models, runs efficiently on GPU, and achieves input-output latency of 0.3 to 1 seconds, marking the first practical diffusion-based solution for online speech enhancement.
Diffusion models are a class of generative models that have been recently used for speech enhancement with remarkable success but are computationally expensive at inference time. Therefore, these models are impractical for processing streaming data in real-time. In this work, we adapt a sliding window diffusion framework to the speech enhancement task. Our approach progressively corrupts speech signals through time, assigning more noise to frames close to the present in a buffer. This approach outputs denoised frames with a delay proportional to the chosen buffer size, enabling a trade-off between performance and latency. Empirical results demonstrate that our method outperforms standard diffusion models and runs efficiently on a GPU, achieving an input-output latency in the order of 0.3 to 1 seconds. This marks the first practical diffusion-based solution for online speech enhancement.