Making Separation-First Multi-Stream Audio Watermarking Feasible via Joint Training
This addresses the need for robust multi-stream audio watermarking in modern audio production, though it is incremental as it builds on existing watermarking and separation methods.
The paper tackles the problem of independently watermarking mixed audio stems and recovering watermarks after separation, showing that a naive approach fails due to separation artifacts, and achieves substantial gains in bit recovery through joint training of the watermark system and separator.
Modern audio is created by mixing stems from different sources, raising the question: can we independently watermark each stem and recover all watermarks after separation? We study a separation-first, multi-stream watermarking framework-embedding distinct information into stems using unique keys but a shared structure, mixing, separating, and decoding from each output. A naive pipeline (robust watermarking + off-the-shelf separation) yields poor bit recovery, showing robustness to generic distortions does not ensure robustness to separation artifacts. To enable this, we jointly train the watermark system and the separator in an end-to-end manner, encouraging the separator to preserve watermark cues while adapting embedding to separation-specific distortions. Experiments on speech+music and vocal+accompaniment mixtures show substantial gains in post-separation recovery while maintaining perceptual quality.