Using growth transform dynamical systems for spatio-temporal data sonification
This work addresses the need for improved sonification methods in human-in-the-loop decision-making, particularly for medical applications like EEG analysis, though it appears incremental as it builds on existing sonification and dynamical system concepts.
The paper tackles the problem of sonifying high-dimensional spatio-temporal data by integrating learning and sonification into a single framework using growth transform dynamical systems, resulting in binaural audio signatures that encode statistical properties and reveal optimization complexity, with demonstrations on EEG data for potential epileptic seizure detection.
Sonification, or encoding information in meaningful audio signatures, has several advantages in augmenting or replacing traditional visualization methods for human-in-the-loop decision-making. Standard sonification methods reported in the literature involve either (i) using only a subset of the variables, or (ii) first solving a learning task on the data and then mapping the output to an audio waveform, which is utilized by the end-user to make a decision. This paper presents a novel framework for sonifying high-dimensional data using a complex growth transform dynamical system model where both the learning (or, more generally, optimization) and the sonification processes are integrated together. Our algorithm takes as input the data and optimization parameters underlying the learning or prediction task and combines it with the psychoacoustic parameters defined by the user. As a result, the proposed framework outputs binaural audio signatures that not only encode some statistical properties of the high-dimensional data but also reveal the underlying complexity of the optimization/learning process. Along with extensive experiments using synthetic datasets, we demonstrate the framework on sonifying Electro-encephalogram (EEG) data with the potential for detecting epileptic seizures in pediatric patients.