Live Orchestral Piano, a system for real-time orchestral music generation
This addresses the challenge of automating orchestration for musicians and composers, but it is incremental as it builds on existing statistical models for music generation.
The paper tackles the problem of generating orchestral music in real-time from piano input by learning mappings from piano scores to orchestration using conditional Restricted Boltzmann Machines, and introduces a system called Live Orchestral Piano that enables real-time orchestration of MIDI keyboard input.
This paper introduces the first system for performing automatic orchestration based on a real-time piano input. We believe that it is possible to learn the underlying regularities existing between piano scores and their orchestrations by notorious composers, in order to automatically perform this task on novel piano inputs. To that end, we investigate a class of statistical inference models called conditional Restricted Boltzmann Machine (cRBM). We introduce a specific evaluation framework for orchestral generation based on a prediction task in order to assess the quality of different models. As prediction and creation are two widely different endeavours, we discuss the potential biases in evaluating temporal generative models through prediction tasks and their impact on a creative system. Finally, we introduce an implementation of the proposed model called Live Orchestral Piano (LOP), which allows to perform real-time projective orchestration of a MIDI keyboard input.