One model to enhance them all: array geometry agnostic multi-channel personalized speech enhancement
This work addresses speech enhancement for video conferencing users by enabling high-quality audio from arbitrary microphone setups, though it is incremental as it builds on existing personalized speech enhancement methods.
The paper tackles the problem of multi-channel personalized speech enhancement for video conferencing by proposing a causal array-geometry-agnostic model that leverages spatial information from microphone arrays, resulting in improved speech quality and automatic speech recognition accuracy compared to geometry-specific models.
With the recent surge of video conferencing tools usage, providing high-quality speech signals and accurate captions have become essential to conduct day-to-day business or connect with friends and families. Single-channel personalized speech enhancement (PSE) methods show promising results compared with the unconditional speech enhancement (SE) methods in these scenarios due to their ability to remove interfering speech in addition to the environmental noise. In this work, we leverage spatial information afforded by microphone arrays to improve such systems' performance further. We investigate the relative importance of speaker embeddings and spatial features. Moreover, we propose a new causal array-geometry-agnostic multi-channel PSE model, which can generate a high-quality enhanced signal from arbitrary microphone geometry. Experimental results show that the proposed geometry agnostic model outperforms the model trained on a specific microphone array geometry in both speech quality and automatic speech recognition accuracy. We also demonstrate the effectiveness of the proposed approach for unseen array geometries.