Tracking Ensemble Performance on Touch-Screens with Gesture Classification and Transition Matrices
This system provides a new way for musicians to track and respond to free-form gestural performances in an ensemble setting, potentially enhancing interactive musical experiences.
This paper introduces a touch-screen interface for tracking ensemble performances by classifying gestures and analyzing transition matrix statistics. The system uses a Random Forest classifier to extract gesture-state sequences, which then inform real-time modifications to musical interfaces on iPads.
We present and evaluate a novel interface for tracking ensemble performances on touch-screens. The system uses a Random Forest classifier to extract touch-screen gestures and transition matrix statistics. It analyses the resulting gesture-state sequences across an ensemble of performers. A series of specially designed iPad apps respond to this real-time analysis of free-form gestural performances with calculated modifications to their musical interfaces. We describe our system and evaluate it through cross-validation and profiling as well as concert experience.