Individualized Conditioning and Negative Distances for Speaker Separation
This work addresses speaker separation for audio processing applications, presenting incremental improvements to existing methods.
The paper tackles speaker separation by proposing two speaker-aware designs: a speaker conditioning network that uses speech samples to generate individualized guidance, and negative distances to penalize non-target voices while positive distances drive outputs toward clean targets. Experiments on LibriMix show the models are effective, though no specific performance numbers are provided.
Speaker separation aims to extract multiple voices from a mixed signal. In this paper, we propose two speaker-aware designs to improve the existing speaker separation solutions. The first model is a speaker conditioning network that integrates speech samples to generate individualized speaker conditions, which then provide informed guidance for a separation module to produce well-separated outputs. The second design aims to reduce non-target voices in the separated speech. To this end, we propose negative distances to penalize the appearance of any non-target voice in the channel outputs, and positive distances to drive the separated voices closer to the clean targets. We explore two different setups, weighted-sum and triplet-like, to integrate these two distances to form a combined auxiliary loss for the separation networks. Experiments conducted on LibriMix demonstrate the effectiveness of our proposed models.