ArtFace: Towards Historical Portrait Face Identification via Model Adaptation
This work addresses the need for automated assistance in art history by improving facial recognition in paintings, though it is incremental as it builds on existing models.
The paper tackled the problem of identifying faces in historical paintings, which is challenging due to domain shift and artistic variations, by fine-tuning foundation models and integrating their embeddings with conventional facial recognition networks, resulting in notable improvements over state-of-the-art methods.
Identifying sitters in historical paintings is a key task for art historians, offering insight into their lives and how they chose to be seen. However, the process is often subjective and limited by the lack of data and stylistic variations. Automated facial recognition is capable of handling challenging conditions and can assist, but while traditional facial recognition models perform well on photographs, they struggle with paintings due to domain shift and high intra-class variation. Artistic factors such as style, skill, intent, and influence from other works further complicate recognition. In this work, we investigate the potential of foundation models to improve facial recognition in artworks. By fine-tuning foundation models and integrating their embeddings with those from conventional facial recognition networks, we demonstrate notable improvements over current state-of-the-art methods. Our results show that foundation models can bridge the gap where traditional methods are ineffective. Paper page at https://www.idiap.ch/paper/artface/