Affective Behavior Analysis using Task-adaptive and AU-assisted Graph Network
This work addresses multi-task learning for affective behavior analysis in-the-wild, which is incremental as it builds on existing competition frameworks with specific architectural improvements.
The authors tackled the Multi-Task Learning Challenge for affective behavior analysis by introducing a pre-trained large model (Dinov2), a task-adaptive block for feature extraction, and an AU-assisted Graph Convolutional Network to leverage correlations between action units, achieving an evaluation measure of 1.2542 on the validation set.
In this paper, we present our solution and experiment result for the Multi-Task Learning Challenge of the 7th Affective Behavior Analysis in-the-wild(ABAW7) Competition. This challenge consists of three tasks: action unit detection, facial expression recognition, and valance-arousal estimation. We address the research problems of this challenge from three aspects: 1)For learning robust visual feature representations, we introduce the pre-trained large model Dinov2. 2) To adaptively extract the required features of eack task, we design a task-adaptive block that performs cross-attention between a set of learnable query vectors and pre-extracted features. 3) By proposing the AU-assisted Graph Convolutional Network(AU-GCN), we make full use of the correlation information between AUs to assist in solving the EXPR and VA tasks. Finally, we achieve the evaluation measure of \textbf{1.2542} on the validation set provided by the organizers.