Mask-Weighted Spatial Likelihood Coding for Speaker-Independent Joint Localization and Mask Estimation
This provides a speaker-independent approach for performance-critical speech separation scenarios, potentially replacing upstream sound source localization systems.
The paper tackles the problem of joint speaker localization and mask estimation in speech separation by proposing mask-weighted spatial likelihood coding, which achieves considerable performance improvements in both tasks compared to baseline encodings optimized for either localization or mask estimation alone.
Due to their robustness and flexibility, neural-driven beamformers are a popular choice for speech separation in challenging environments with a varying amount of simultaneous speakers alongside noise and reverberation. Time-frequency masks and relative directions of the speakers regarding a fixed spatial grid can be used to estimate the beamformer's parameters. To some degree, speaker-independence is achieved by ensuring a greater amount of spatial partitions than speech sources. In this work, we analyze how to encode both mask and positioning into such a grid to enable joint estimation of both quantities. We propose mask-weighted spatial likelihood coding and show that it achieves considerable performance in both tasks compared to baseline encodings optimized for either localization or mask estimation. In the same setup, we demonstrate superiority for joint estimation of both quantities. Conclusively, we propose a universal approach which can replace an upstream sound source localization system solely by adapting the training framework, making it highly relevant in performance-critical scenarios.