Time-Frequency Trade-offs for Audio Source Separation with Binary Masks
This work provides insights for improving audio source separation in machine audition, though it is incremental as it builds on existing binary mask methods.
The study investigated how the STFT window size affects binary mask audio source separation quality for speech and music, finding that optimal window sizes vary depending on the source types.
The short-time Fourier transform (STFT) provides the foundation of binary-mask based audio source separation approaches. In computing a spectrogram, the STFT window size parameterizes the trade-off between time and frequency resolution. However, it is not yet known how this parameter affects the operation of the binary mask in terms of separation quality for real-world signals such as speech or music. Here, we demonstrate that the trade-off between time and frequency in the STFT, used to perform ideal binary mask separation, depends upon the types of source that are to be separated. In particular, we demonstrate that different window sizes are optimal for separating different combinations of speech and musical signals. Our findings have broad implications for machine audition and machine learning in general.