Audio Spectrogram Representations for Processing with Convolutional Neural Networks
It addresses the problem of selecting effective audio representations for neural network processing, particularly in style transfer, but is incremental as it reviews existing methods without introducing new results.
The paper reviews various audio representations for neural networks, focusing on spectrograms for style transfer applications.
One of the decisions that arise when designing a neural network for any application is how the data should be represented in order to be presented to, and possibly generated by, a neural network. For audio, the choice is less obvious than it seems to be for visual images, and a variety of representations have been used for different applications including the raw digitized sample stream, hand-crafted features, machine discovered features, MFCCs and variants that include deltas, and a variety of spectral representations. This paper reviews some of these representations and issues that arise, focusing particularly on spectrograms for generating audio using neural networks for style transfer.