Can We Estimate Purchase Intention Based on Zero-shot Speech Emotion Recognition?
This work addresses the challenge of recognizing undefined emotions in speech, such as purchase intention, which is incremental as it builds on existing CLAP frameworks.
The paper tackles the problem of zero-shot speech emotion recognition for estimating bipolar emotions like purchase intention, achieving results comparable to supervised learning models.
This paper proposes a zero-shot speech emotion recognition (SER) method that estimates emotions not previously defined in the SER model training. Conventional methods are limited to recognizing emotions defined by a single word. Moreover, we have the motivation to recognize unknown bipolar emotions such as ``I want to buy - I do not want to buy.'' In order to allow the model to define classes using sentences freely and to estimate unknown bipolar emotions, our proposed method expands upon the contrastive language-audio pre-training (CLAP) framework by introducing multi-class and multi-task settings. We also focus on purchase intention as a bipolar emotion and investigate the model's performance to zero-shot estimate it. This study is the first attempt to estimate purchase intention from speech directly. Experiments confirm that the results of zero-shot estimation by the proposed method are at the same level as those of the model trained by supervised learning.