Polyphonic Pitch Tracking with Deep Layered Learning
This work addresses the problem of accurate polyphonic pitch tracking for music information retrieval, representing an incremental improvement with strong performance gains.
The paper tackles polyphonic pitch tracking by developing a deep layered learning system that uses cascading neural networks for framewise fundamental frequency estimation, followed by onset/offset detection and iterative note refinement, achieving state-of-the-art results on four public datasets including MAPS and Bach10.
This paper presents a polyphonic pitch tracking system able to extract both framewise and note-based estimates from audio. The system uses several artificial neural networks in a deep layered learning setup. First, cascading networks are applied to a spectrogram for framewise fundamental frequency (f0) estimation. A sparse receptive field is learned by the first network and then used as a filter kernel for parameter sharing throughout the system. The f0 activations are connected across time to extract pitch contours. These contours define a framework within which subsequent networks perform onset and offset detection, operating across both time and smaller pitch fluctuations at the same time. As input, the networks use, e.g., variations of latent representations from the f0 estimation network. Finally, incorrect tentative notes are removed one by one in an iterative procedure that allows a network to classify notes within an accurate context. The system was evaluated on four public test sets: MAPS, Bach10, TRIOS, and the MIREX Woodwind quintet, and performed state-of-the-art results for all four datasets. It performs well across all subtasks: f0, pitched onset, and pitched offset tracking.