Deep Predictive Models in Interactive Music
This work addresses the problem of enhancing musical performance and interaction in digital instruments for musicians and designers, but it is incremental as it reviews and motivates existing applications rather than introducing new methods.
The paper investigates how digital musical instruments (DMIs) use predictive machine learning models to assist users by predicting unknown states in musical processes, such as within instruments, at performer level, and in ensembles, and discusses how deep learning advances enable data-driven models with long memory to address challenges in interactive music systems.
Musical performance requires prediction to operate instruments, to perform in groups and to improvise. In this paper, we investigate how a number of digital musical instruments (DMIs), including two of our own, have applied predictive machine learning models that assist users by predicting unknown states of musical processes. We characterise these predictions as focussed within a musical instrument, at the level of individual performers, and between members of an ensemble. These models can connect to existing frameworks for DMI design and have parallels in the cognitive predictions of human musicians. We discuss how recent advances in deep learning highlight the role of prediction in DMIs, by allowing data-driven predictive models with a long memory of past states. The systems we review are used to motivate musical use-cases where prediction is a necessary component, and to highlight a number of challenges for DMI designers seeking to apply deep predictive models in interactive music systems of the future.