Burst2Vec: An Adversarial Multi-Task Approach for Predicting Emotion, Age, and Origin from Vocal Bursts
This work addresses the challenge of multi-task prediction from vocal bursts for applications in affective computing and human-computer interaction, though it appears incremental as it builds on existing pre-trained representations and adversarial methods.
The paper tackled the problem of predicting emotion, age, and origin from vocal bursts by developing Burst2Vec, a multi-task learning approach that uses pre-trained speech representations and adversarial training, achieving a 30% performance gain over baselines and winning the ICML ExVo 2022 Multi-Task Challenge.
We present Burst2Vec, our multi-task learning approach to predict emotion, age, and origin (i.e., native country/language) from vocal bursts. Burst2Vec utilises pre-trained speech representations to capture acoustic information from raw waveforms and incorporates the concept of model debiasing via adversarial training. Our models achieve a relative 30 % performance gain over baselines using pre-extracted features and score the highest amongst all participants in the ICML ExVo 2022 Multi-Task Challenge.