Adapt then Unlearn: Exploring Parameter Space Semantics for Unlearning in Generative Adversarial Networks
This addresses privacy and regulatory compliance needs for generative models, offering a novel solution for unlearning in high-fidelity GANs like StyleGAN, though it is incremental in applying unlearning concepts to this specific domain.
The paper tackles the problem of preventing generative adversarial networks (GANs) from producing outputs with undesired features when the training data is inaccessible, proposing a two-stage 'Adapt-then-Unlearn' method that maintains sample quality while effectively unlearning these features, as validated on datasets like MNIST, AFHQ, and CelebA-HQ.
Owing to the growing concerns about privacy and regulatory compliance, it is desirable to regulate the output of generative models. To that end, the objective of this work is to prevent the generation of outputs containing undesired features from a pre-trained Generative Adversarial Network (GAN) where the underlying training data set is inaccessible. Our approach is inspired by the observation that the parameter space of GANs exhibits meaningful directions that can be leveraged to suppress specific undesired features. However, such directions usually result in the degradation of the quality of generated samples. Our proposed two-stage method, known as 'Adapt-then-Unlearn,' excels at unlearning such undesirable features while also maintaining the quality of generated samples. In the initial stage, we adapt a pre-trained GAN on a set of negative samples (containing undesired features) provided by the user. Subsequently, we train the original pre-trained GAN using positive samples, along with a repulsion regularizer. This regularizer encourages the learned model parameters to move away from the parameters of the adapted model (first stage) while not degrading the generation quality. We provide theoretical insights into the proposed method. To the best of our knowledge, our approach stands as the first method addressing unlearning within the realm of high-fidelity GANs (such as StyleGAN). We validate the effectiveness of our method through comprehensive experiments, encompassing both class-level unlearning on the MNIST and AFHQ dataset and feature-level unlearning tasks on the CelebA-HQ dataset. Our code and implementation is available at: https://github.com/atriguha/Adapt_Unlearn.