Task-Specific Feature Fusion Method for Multi-Task Affective Behavior Analysis
For participants in the ABAW11 challenge, this work provides a simple and effective strategy for multi-task learning, though it is incremental as it systematically compares existing techniques.
The authors tackle multi-task affective behavior analysis by proposing a task-specific feature fusion method that selects different fusion and prediction strategies for each task (valence-arousal, expressions, action units) rather than a shared architecture. On the ABAW11 validation set, they achieve an EXPR macro-F1 of 0.4222, AU macro-F1 of 0.5402, and mean VA CCC of 0.6717, with an overall score of 1.6341.
The 11th Affective Behavior Analysis in-the-wild (ABAW11) Multi-Task Learning Challenge requires a unified system to predict valence-arousal, categorical expressions, and facial action units from the official s-Aff-Wild2 images. Although these tasks are naturally related through facial behavior, our validation experiments show that they benefit from different visual features, temporal processing strategies, fusion mechanisms, and calibration procedures. In this paper, we study task-adaptive feature fusion for ABAW11 multi-task affective behavior analysis. We first adapt two pretrained visual backbones, DINOv2 ViT-L and DINOv3 ConvNeXt-base, on an external expression-oriented facial image set and then freeze them to extract complementary frame-level features from the official ABAW11 data. On top of these frozen features, we systematically compare frame-level prediction heads, temporal convolutional heads, post-hoc temporal smoothing, LightGBM models, feature concatenation, gated fusion, residual fusion, late logit fusion, threshold calibration, and shared MTL structures. The final system selects task-specific fusion and prediction strategies rather than forcing all tasks to share a single architecture. On the ABAW11 validation set, the selected system achieves an EXPR macro-F1 of 0.4222, an AU macro-F1 of 0.5402, and a mean VA CCC of 0.6717, resulting in an overall validation score of 1.6341. The results suggest that task-adaptive fusion of frozen visual features is a simple and effective strategy for ABAW-style multi-task affective behavior analysis.