Rethinking the Role of Feature Engineering and Learning Strategies in Few-Shot Hidden Emotion Recognition
For researchers in affective computing and video understanding, this work addresses the challenge of recognizing weak and sparse implicit emotions in long videos, though the improvements are incremental over a previous winning solution.
The paper presents a compact multi-modal temporal modeling framework for few-shot hidden emotion recognition, achieving first place in Track 3 of the 4th EI-MIGA-IJCAI Challenge with a test accuracy of 0.76923. It integrates multi-source features and proposes a cross-attention mechanism to capture dynamic micro-motions while suppressing static biases.
In this paper, we present the solution developed by our team, XInsight Lab, which achieved first place in Track 3 of the 4th EI-MIGA-IJCAI Challenge with a test accuracy of 0.76923. To address the challenge of weak and sparse implicit emotion evidence in long videos, this paper extends the winning solution from the previous competition and proposes a compact multi-modal temporal modeling framework. The framework integrates and evaluates the effects of multi-source features, including 2D/3D skeletons, facial expression Blendshapes, DINOv2/v3 vision foundation models, X-CLIP video features, and Gemini semantic priors. Architecturally, we propose a cross-attention mechanism that utilizes static pose features, denoted as Base, as the Query and dynamic micro-motion differential features, denoted as Offset, as the Key and Value. By capturing local relative velocities, this mechanism eliminates static biases related to individual body shape and identity. Concurrently, an adaptive pooling method based on Multiple Instance Learning is employed to extract instantaneous emotions while suppressing background noise in long sequences. Finally, the paper reveals the representation collapse phenomenon of general vision foundation models in micro-dynamic tasks, and analyzes the underlying mechanisms where networks fall into public-leaderboard-driven pseudo-generalization due to shortcut learning and rote memorization.