Real Time Speech Enhancement in the Waveform Domain
This addresses the problem of real-time speech enhancement for applications like communication or hearing aids, though it is incremental as it builds on existing encoder-decoder architectures.
The authors tackled real-time speech enhancement by developing a causal model that processes raw waveforms and runs on a laptop CPU, achieving state-of-the-art performance on standard benchmarks by removing background noise and reverb.
We present a causal speech enhancement model working on the raw waveform that runs in real-time on a laptop CPU. The proposed model is based on an encoder-decoder architecture with skip-connections. It is optimized on both time and frequency domains, using multiple loss functions. Empirical evidence shows that it is capable of removing various kinds of background noise including stationary and non-stationary noises, as well as room reverb. Additionally, we suggest a set of data augmentation techniques applied directly on the raw waveform which further improve model performance and its generalization abilities. We perform evaluations on several standard benchmarks, both using objective metrics and human judgements. The proposed model matches state-of-the-art performance of both causal and non causal methods while working directly on the raw waveform.