Audio inpainting of music by means of neural networks
This work addresses audio inpainting for music, which is an incremental improvement in a domain-specific application.
The study tackled the problem of restoring missing audio content in music using deep neural networks, achieving significant performance improvements over a linear predictive coding reference method for gaps of tens of milliseconds.
We studied the ability of deep neural networks (DNNs) to restore missing audio content based on its context, a process usually referred to as audio inpainting. We focused on gaps in the range of tens of milliseconds. The proposed DNN structure was trained on audio signals containing music and musical instruments, separately, with 64-ms long gaps. The input to the DNN was the context, i.e., the signal surrounding the gap, transformed into time-frequency (TF) coefficients. Our results were compared to those obtained from a reference method based on linear predictive coding (LPC). For music, our DNN significantly outperformed the reference method, demonstrating a generally good usability of the proposed DNN structure for inpainting complex audio signals like music.