General Facial Representation Learning in a Visual-Linguistic Manner
This work addresses the need for better facial representations in computer vision, offering incremental improvements over existing methods for researchers and practitioners in face analysis.
The paper tackles the problem of learning a universal facial representation to improve all face analysis tasks by introducing FaRL, a visual-linguistic framework that combines contrastive learning and masked image modeling, achieving state-of-the-art results on tasks like face parsing and alignment.
How to learn a universal facial representation that boosts all face analysis tasks? This paper takes one step toward this goal. In this paper, we study the transfer performance of pre-trained models on face analysis tasks and introduce a framework, called FaRL, for general Facial Representation Learning in a visual-linguistic manner. On one hand, the framework involves a contrastive loss to learn high-level semantic meaning from image-text pairs. On the other hand, we propose exploring low-level information simultaneously to further enhance the face representation, by adding a masked image modeling. We perform pre-training on LAION-FACE, a dataset containing large amount of face image-text pairs, and evaluate the representation capability on multiple downstream tasks. We show that FaRL achieves better transfer performance compared with previous pre-trained models. We also verify its superiority in the low-data regime. More importantly, our model surpasses the state-of-the-art methods on face analysis tasks including face parsing and face alignment.