miditok: A Python package for MIDI file tokenization
This provides a practical tool for researchers and developers working on symbolic music processing, but it is incremental as it consolidates existing methods.
The authors tackled the lack of a unified and flexible tokenization tool for symbolic music in NLP applications, resulting in MidiTok, an open-source Python package that supports popular tokenizations under a single API.
Recent progress in natural language processing has been adapted to the symbolic music modality. Language models, such as Transformers, have been used with symbolic music for a variety of tasks among which music generation, modeling or transcription, with state-of-the-art performances. These models are beginning to be used in production products. To encode and decode music for the backbone model, they need to rely on tokenizers, whose role is to serialize music into sequences of distinct elements called tokens. MidiTok is an open-source library allowing to tokenize symbolic music with great flexibility and extended features. It features the most popular music tokenizations, under a unified API. It is made to be easily used and extensible for everyone.