Machine Learning for a Music Glove Instrument
This work presents a data-driven approach for digital musical instruments, which is incremental as it applies existing machine learning methods to a new application domain.
The researchers tackled the problem of generating musical note sequences from hand motions using a sensor-equipped glove, achieving a system that can predict notes on any surface based on learned sensor inputs from piano training.
A music glove instrument equipped with force sensitive, flex and IMU sensors is trained on an electric piano to learn note sequences based on a time series of sensor inputs. Once trained, the glove is used on any surface to generate the sequence of notes most closely related to the hand motion. The data is collected manually by a performer wearing the glove and playing on an electric keyboard. The feature space is designed to account for the key hand motion, such as the thumb-under movement. Logistic regression along with bayesian belief networks are used learn the transition probabilities from one note to another. This work demonstrates a data-driven approach for digital musical instruments in general.