Emotions as Ambiguity-aware Ordinal Representations
This work addresses the challenge of modeling ambiguous and dynamic emotions for affective computing, representing an incremental improvement over existing methods.
The paper tackled the problem of continuous emotion recognition by addressing the ambiguity and temporal dynamics of emotions, introducing ambiguity-aware ordinal representations that outperform conventional models on unbounded labels with higher CCC and SDA scores, and excel in SDA for bounded traces.
Emotions are inherently ambiguous and dynamic phenomena, yet existing continuous emotion recognition approaches either ignore their ambiguity or treat ambiguity as an independent and static variable over time. Motivated by this gap in the literature, in this paper we introduce ambiguity-aware ordinal emotion representations, a novel framework that captures both the ambiguity present in emotion annotation and the inherent temporal dynamics of emotional traces. Specifically, we propose approaches that model emotion ambiguity through its rate of change. We evaluate our framework on two affective corpora -- RECOLA and GameVibe -- testing our proposed approaches on both bounded (arousal, valence) and unbounded (engagement) continuous traces. Our results demonstrate that ordinal representations outperform conventional ambiguity-aware models on unbounded labels, achieving the highest Concordance Correlation Coefficient (CCC) and Signed Differential Agreement (SDA) scores, highlighting their effectiveness in modeling the traces' dynamics. For bounded traces, ordinal representations excel in SDA, revealing their superior ability to capture relative changes of annotated emotion traces.