Adv-CPG: A Customized Portrait Generation Framework with Facial Adversarial Attacks
This addresses privacy protection for users of portrait generation systems, though it is incremental as it builds on existing CPG methods by adding adversarial attacks.
The paper tackles the problem of preventing malicious face recognition systems from tracking and misusing generated portraits in Customized Portrait Generation (CPG) by proposing Adv-CPG, a framework that integrates facial adversarial attacks, achieving an average attack success rate 28.1% and 2.86% higher than state-of-the-art methods.
Recent Customized Portrait Generation (CPG) methods, taking a facial image and a textual prompt as inputs, have attracted substantial attention. Although these methods generate high-fidelity portraits, they fail to prevent the generated portraits from being tracked and misused by malicious face recognition systems. To address this, this paper proposes a Customized Portrait Generation framework with facial Adversarial attacks (Adv-CPG). Specifically, to achieve facial privacy protection, we devise a lightweight local ID encryptor and an encryption enhancer. They implement progressive double-layer encryption protection by directly injecting the target identity and adding additional identity guidance, respectively. Furthermore, to accomplish fine-grained and personalized portrait generation, we develop a multi-modal image customizer capable of generating controlled fine-grained facial features. To the best of our knowledge, Adv-CPG is the first study that introduces facial adversarial attacks into CPG. Extensive experiments demonstrate the superiority of Adv-CPG, e.g., the average attack success rate of the proposed Adv-CPG is 28.1% and 2.86% higher compared to the SOTA noise-based attack methods and unconstrained attack methods, respectively.