Test-Time Augmentation for Pose-invariant Face Recognition
This addresses the practical challenge of pose-invariant face recognition for security and identification systems, though it is an incremental improvement over existing augmentation approaches.
The paper tackles the problem of face recognition across different head poses by proposing Pose-TTA, a test-time augmentation method that generates matching side-profile images for comparison instead of frontalizing faces, which consistently improves inference performance across diverse datasets and pre-trained models without requiring retraining.
The goal of this paper is to enhance face recognition performance by augmenting head poses during the testing phase. Existing methods often rely on training on frontalised images or learning pose-invariant representations, yet both approaches typically require re-training and testing for each dataset, involving a substantial amount of effort. In contrast, this study proposes Pose-TTA, a novel approach that aligns faces at inference time without additional training. To achieve this, we employ a portrait animator that transfers the source image identity into the pose of a driving image. Instead of frontalising a side-profile face -- which can introduce distortion -- Pose-TTA generates matching side-profile images for comparison, thereby reducing identity information loss. Furthermore, we propose a weighted feature aggregation strategy to address any distortions or biases arising from the synthetic data, thus enhancing the reliability of the augmented images. Extensive experiments on diverse datasets and with various pre-trained face recognition models demonstrate that Pose-TTA consistently improves inference performance. Moreover, our method is straightforward to integrate into existing face recognition pipelines, as it requires no retraining or fine-tuning of the underlying recognition models.