Evaluating Multimodal Large Language Models for Heterogeneous Face Recognition
This work addresses the problem of assessing MLLMs for biometric applications like HFR, revealing critical gaps that could impact deployment in security or identification systems, though it is incremental as it focuses on evaluation rather than proposing new methods.
The paper systematically evaluated state-of-the-art multimodal large language models (MLLMs) for heterogeneous face recognition (HFR) across cross-modality scenarios like VIS-NIR and VIS-THERMAL, finding substantial performance gaps compared to classical systems, with metrics such as Equal Error Rate (EER) and True Accept Rate (TAR) indicating limitations under challenging conditions.
Multimodal Large Language Models (MLLMs) have recently demonstrated strong performance on a wide range of vision-language tasks, raising interest in their potential use for biometric applications. In this paper, we conduct a systematic evaluation of state-of-the-art MLLMs for heterogeneous face recognition (HFR), where enrollment and probe images are from different sensing modalities, including visual (VIS), near infrared (NIR), short-wave infrared (SWIR), and thermal camera. We benchmark multiple open-source MLLMs across several cross-modality scenarios, including VIS-NIR, VIS-SWIR, and VIS-THERMAL face recognition. The recognition performance of MLLMs is evaluated using biometric protocols and based on different metrics, including Acquire Rate, Equal Error Rate (EER), and True Accept Rate (TAR). Our results reveal substantial performance gaps between MLLMs and classical face recognition systems, particularly under challenging cross-spectral conditions, in spite of recent advances in MLLMs. Our findings highlight the limitations of current MLLMs for HFR and also the importance of rigorous biometric evaluation when considering their deployment in face recognition systems.