Two Web Toolkits for Multimodal Piano Performance Dataset Acquisition and Fingering Annotation
This addresses the laborious data acquisition problem for researchers in multimodal piano performance analysis, though it is incremental as it builds on existing tools.
The researchers tackled the bottleneck of acquiring large-scale multimodal piano performance data by developing an integrated web toolkit with two GUIs, PiaRec for synchronized data acquisition and ASDF for fingering annotation, which streamlines dataset creation.
Piano performance is a multimodal activity that intrinsically combines physical actions with the acoustic rendition. Despite growing research interest in analyzing the multimodal nature of piano performance, the laborious process of acquiring large-scale multimodal data remains a significant bottleneck, hindering further progress in this field. To overcome this barrier, we present an integrated web toolkit comprising two graphical user interfaces (GUIs): (i) PiaRec, which supports the synchronized acquisition of audio, video, MIDI, and performance metadata. (ii) ASDF, which enables the efficient annotation of performer fingering from the visual data. Collectively, this system can streamline the acquisition of multimodal piano performance datasets.