A Cross-Domain Approach for Continuous Impression Recognition from Dyadic Audio-Visual-Physio Signals
This work addresses impression recognition for affective computing and social signal processing, offering an incremental improvement by integrating receiver responses.
The paper tackled impression recognition by incorporating both emitter and receiver signals using a cross-domain architecture, achieving a concordance correlation coefficient of 0.770 in competence and 0.748 in warmth dimensions.
The impression we make on others depends not only on what we say, but also, to a large extent, on how we say it. As a sub-branch of affective computing and social signal processing, impression recognition has proven critical in both human-human conversations and spoken dialogue systems. However, most research has studied impressions only from the signals expressed by the emitter, ignoring the response from the receiver. In this paper, we perform impression recognition using a proposed cross-domain architecture on the dyadic IMPRESSION dataset. This improved architecture makes use of cross-domain attention and regularization. The cross-domain attention consists of intra- and inter-attention mechanisms, which capture intra- and inter-domain relatedness, respectively. The cross-domain regularization includes knowledge distillation and similarity enhancement losses, which strengthen the feature connections between the emitter and receiver. The experimental evaluation verified the effectiveness of our approach. Our approach achieved a concordance correlation coefficient of 0.770 in competence dimension and 0.748 in warmth dimension.