A Weakly Supervised Approach to Emotion-change Prediction and Improved Mood Inference
This work addresses the understudied mood-emotion interplay for affective computing researchers, but it is incremental as it builds on prior work.
The paper tackled the problem of mood inference in affective computing by incorporating emotion-change information without annotated labels, resulting in improved mood prediction for long video clips, as shown by experiments comparing unimodal and multimodal models.
Whilst a majority of affective computing research focuses on inferring emotions, examining mood or understanding the \textit{mood-emotion interplay} has received significantly less attention. Building on prior work, we (a) deduce and incorporate emotion-change ($Δ$) information for inferring mood, without resorting to annotated labels, and (b) attempt mood prediction for long duration video clips, in alignment with the characterisation of mood. We generate the emotion-change ($Δ$) labels via metric learning from a pre-trained Siamese Network, and use these in addition to mood labels for mood classification. Experiments evaluating \textit{unimodal} (training only using mood labels) vs \textit{multimodal} (training using mood plus $Δ$ labels) models show that mood prediction benefits from the incorporation of emotion-change information, emphasising the importance of modelling the mood-emotion interplay for effective mood inference.