Learning audio representations via phase prediction
This addresses audio representation learning for various downstream tasks, but it is incremental as it builds on existing self-supervised and phase prediction methods.
The paper tackles the problem of learning audio representations by predicting the phase from magnitude in a self-supervised task, resulting in representations that generalize across downstream tasks and partially bridge the gap with supervised models, while also reducing Griffin-Lim algorithm iterations for waveform reconstruction.
We learn audio representations by solving a novel self-supervised learning task, which consists of predicting the phase of the short-time Fourier transform from its magnitude. A convolutional encoder is used to map the magnitude spectrum of the input waveform to a lower dimensional embedding. A convolutional decoder is then used to predict the instantaneous frequency (i.e., the temporal rate of change of the phase) from such embedding. To evaluate the quality of the learned representations, we evaluate how they transfer to a wide variety of downstream audio tasks. Our experiments reveal that the phase prediction task leads to representations that generalize across different tasks, partially bridging the gap with fully-supervised models. In addition, we show that the predicted phase can be used as initialization of the Griffin-Lim algorithm, thus reducing the number of iterations needed to reconstruct the waveform in the time domain.