Mind the beat: detecting audio onsets from EEG recordings of music listening
This addresses a novel problem in neuroscience and music processing, but it is incremental as it applies existing deep learning methods to new data without a pre-existing benchmark.
The paper tackles the problem of predicting audio event onsets from EEG recordings during music listening, using deep learning models like FCN and RNN, and finds that the RNN generalizes better than other methods, though no specific numerical results are provided.
We propose a deep learning approach to predicting audio event onsets in electroencephalogram (EEG) recorded from users as they listen to music. We use a publicly available dataset containing ten contemporary songs and concurrently recorded EEG. We generate a sequence of onset labels for the songs in our dataset and trained neural networks (a fully connected network (FCN) and a recurrent neural network (RNN)) to parse one second windows of input EEG to predict one second windows of onsets in the audio. We compare our RNN network to both the standard spectral-flux based novelty function and the FCN. We find that our RNN was able to produce results that reflected its ability to generalize better than the other methods. Since there are no pre-existing works on this topic, the numbers presented in this paper may serve as useful benchmarks for future approaches to this research problem.