Cross-Cultural and Cultural-Specific Production and Perception of Facial Expressions of Emotion in the Wild
This addresses the need for foundational data to design emotion recognition technology that works across diverse cultural contexts, though it is incremental as it provides empirical evidence rather than a new method.
The paper tackled the problem of identifying which facial expressions are commonly used to convey emotions across and within cultures in real-world settings, finding that only 35 out of 16,384 possible configurations transmit emotive information across cultures and only 8 within specific cultures, with variations in perception of emotion categories, valence, and arousal.
Automatic recognition of emotion from facial expressions is an intense area of research, with a potentially long list of important application. Yet, the study of emotion requires knowing which facial expressions are used within and across cultures in the wild, not in controlled lab conditions; but such studies do not exist. Which and how many cross-cultural and cultural-specific facial expressions do people commonly use? And, what affect variables does each expression communicate to observers? If we are to design technology that understands the emotion of users, we need answers to these two fundamental questions. In this paper, we present the first large-scale study of the production and visual perception of facial expressions of emotion in the wild. We find that of the 16,384 possible facial configurations that people can theoretically produce, only 35 are successfully used to transmit emotive information across cultures, and only 8 within a smaller number of cultures. Crucially, we find that visual analysis of cross-cultural expressions yields consistent perception of emotion categories and valence, but not arousal. In contrast, visual analysis of cultural-specific expressions yields consistent perception of valence and arousal, but not of emotion categories. Additionally, we find that the number of expressions used to communicate each emotion is also different, e.g., 17 expressions transmit happiness, but only 1 is used to convey disgust.