WHAM!: Extending Speech Separation to Noisy Environments
This work addresses the cocktail party problem for speech processing by moving towards more realistic scenarios, though it is incremental as it extends existing methods to noisy data.
The paper tackled the problem of speech separation in noisy environments by creating the WHAM! dataset with real ambient noise, and found that while noise reduces performance, most approaches still show substantial gains compared to noisy signals.
Recent progress in separating the speech signals from multiple overlapping speakers using a single audio channel has brought us closer to solving the cocktail party problem. However, most studies in this area use a constrained problem setup, comparing performance when speakers overlap almost completely, at artificially low sampling rates, and with no external background noise. In this paper, we strive to move the field towards more realistic and challenging scenarios. To that end, we created the WSJ0 Hipster Ambient Mixtures (WHAM!) dataset, consisting of two speaker mixtures from the wsj0-2mix dataset combined with real ambient noise samples. The samples were collected in coffee shops, restaurants, and bars in the San Francisco Bay Area, and are made publicly available. We benchmark various speech separation architectures and objective functions to evaluate their robustness to noise. While separation performance decreases as a result of noise, we still observe substantial gains relative to the noisy signals for most approaches.