Vec2Face: Unveil Human Faces from their Blackbox Features in Face Recognition
This addresses the challenge of interpreting blackbox face recognition systems for security and privacy applications, though it is incremental as it builds on existing generative methods.
The paper tackles the problem of reconstructing face images from blackbox face recognition features by proposing DiBiGAN, a generative model with bijective metric learning and distillation, achieving realistic synthesis and identity preservation on datasets like CelebA and LFW.
Unveiling face images of a subject given his/her high-level representations extracted from a blackbox Face Recognition engine is extremely challenging. It is because the limitations of accessible information from that engine including its structure and uninterpretable extracted features. This paper presents a novel generative structure with Bijective Metric Learning, namely Bijective Generative Adversarial Networks in a Distillation framework (DiBiGAN), for synthesizing faces of an identity given that person's features. In order to effectively address this problem, this work firstly introduces a bijective metric so that the distance measurement and metric learning process can be directly adopted in image domain for an image reconstruction task. Secondly, a distillation process is introduced to maximize the information exploited from the blackbox face recognition engine. Then a Feature-Conditional Generator Structure with Exponential Weighting Strategy is presented for a more robust generator that can synthesize realistic faces with ID preservation. Results on several benchmarking datasets including CelebA, LFW, AgeDB, CFP-FP against matching engines have demonstrated the effectiveness of DiBiGAN on both image realism and ID preservation properties.