Learning an Ensemble Token from Task-driven Priors in Facial Analysis
This work addresses the challenge of task-specific feature variations in facial analysis, offering an incremental improvement for researchers and practitioners in computer vision.
The paper tackles the problem of preserving unified feature representation in single-task learning for facial analysis by introducing ET-Fuser, a method that learns an ensemble token using attention mechanisms based on task priors from pre-trained models, resulting in statistically significant improvements in feature representations across various facial analysis tasks.
Facial analysis exhibits task-specific feature variations. While Convolutional Neural Networks (CNNs) have enabled the fine-grained representation of spatial information, Vision Transformers (ViTs) have facilitated the representation of semantic information at the patch level. Although the generalization of conventional methodologies has advanced visual interpretability, there remains paucity of research that preserves the unified feature representation on single task learning during the training process. In this work, we introduce ET-Fuser, a novel methodology for learning ensemble token by leveraging attention mechanisms based on task priors derived from pre-trained models for facial analysis. Specifically, we propose a robust prior unification learning method that generates a ensemble token within a self-attention mechanism, which shares the mutual information along the pre-trained encoders. This ensemble token approach offers high efficiency with negligible computational cost. Our results show improvements across a variety of facial analysis, with statistically significant enhancements observed in the feature representations.