TensorFlow Audio Models in Essentia
This work addresses the need for efficient and extensible audio analysis tools for researchers and developers in music information retrieval, though it is incremental as it builds on existing libraries and models.
The authors integrated TensorFlow into the Essentia audio analysis library to enable predictions with pre-trained deep learning models, focusing on flexibility and real-time inference, and demonstrated its potential by providing and evaluating state-of-the-art CNN models for music tagging and classification.
Essentia is a reference open-source C++/Python library for audio and music analysis. In this work, we present a set of algorithms that employ TensorFlow in Essentia, allow predictions with pre-trained deep learning models, and are designed to offer flexibility of use, easy extensibility, and real-time inference. To show the potential of this new interface with TensorFlow, we provide a number of pre-trained state-of-the-art music tagging and classification CNN models. We run an extensive evaluation of the developed models. In particular, we assess the generalization capabilities in a cross-collection evaluation utilizing both external tag datasets as well as manual annotations tailored to the taxonomies of our models.