Modality Agnostic Heterogeneous Face Recognition with Switch Style Modulators
This addresses the challenge of cross-modal authentication for face recognition systems, offering a more flexible solution compared to modality-specific approaches, though it is incremental in improving existing HFR methods.
The paper tackles the problem of heterogeneous face recognition (HFR) across multiple modalities without needing explicit target modality labels, achieving this by introducing Switch Style Modulation Blocks (SSMB) that adaptively reduce domain gaps and enable modality-agnostic inference.
Heterogeneous Face Recognition (HFR) systems aim to enhance the capability of face recognition in challenging cross-modal authentication scenarios. However, the significant domain gap between the source and target modalities poses a considerable challenge for cross-domain matching. Existing literature primarily focuses on developing HFR approaches for specific pairs of face modalities, necessitating the explicit training of models for each source-target combination. In this work, we introduce a novel framework designed to train a modality-agnostic HFR method capable of handling multiple modalities during inference, all without explicit knowledge of the target modality labels. We achieve this by implementing a computationally efficient automatic routing mechanism called Switch Style Modulation Blocks (SSMB) that trains various domain expert modulators which transform the feature maps adaptively reducing the domain gap. Our proposed SSMB can be trained end-to-end and seamlessly integrated into pre-trained face recognition models, transforming them into modality-agnostic HFR models. We have performed extensive evaluations on HFR benchmark datasets to demonstrate its effectiveness. The source code and protocols will be made publicly available.