DeepFilterNet2: Towards Real-Time Speech Enhancement on Embedded Devices for Full-Band Audio
This work addresses the challenge of deploying efficient speech enhancement on resource-constrained embedded devices, representing an incremental improvement over prior methods.
The authors tackled the problem of real-time speech enhancement on embedded devices for full-band audio by extending DeepFilterNet with optimizations in training, data augmentation, and network structure, achieving state-of-the-art performance with a real-time factor of 0.04 on a Core-i5 CPU.
Deep learning-based speech enhancement has seen huge improvements and recently also expanded to full band audio (48 kHz). However, many approaches have a rather high computational complexity and require big temporal buffers for real time usage e.g. due to temporal convolutions or attention. Both make those approaches not feasible on embedded devices. This work further extends DeepFilterNet, which exploits harmonic structure of speech allowing for efficient speech enhancement (SE). Several optimizations in the training procedure, data augmentation, and network structure result in state-of-the-art SE performance while reducing the real-time factor to 0.04 on a notebook Core-i5 CPU. This makes the algorithm applicable to run on embedded devices in real-time. The DeepFilterNet framework can be obtained under an open source license.