Did somebody say "Gest-IT"? A pilot exploration of multimodal data management
This work addresses the problem of multimodal data management for researchers studying gesture variation in specific conversational contexts, but it is incremental as it builds on existing annotation and corpus methods.
The paper tackled the challenge of constructing and managing a multimodal corpus for analyzing gesture patterns in conversations between sighted and visually impaired individuals, resulting in the creation of the Gest-IT resource with a three-layer annotation system and a unified CoNLL-U corpus.
The paper presents a pilot exploration of the construction, management and analysis of a multimodal corpus. Through a three-layer annotation that provides orthographic, prosodic, and gestural transcriptions, the Gest-IT resource allows to investigate the variation of gesture-making patterns in conversations between sighted people and people with visual impairment. After discussing the transcription methods and technical procedures employed in our study, we propose a unified CoNLL-U corpus and indicate our future steps