Sharpness-Aware Machine Unlearning
This work addresses the challenge of efficiently removing specific data from trained models while maintaining performance, which is crucial for privacy and compliance in AI systems, representing an incremental improvement over existing unlearning methods.
The paper tackles the problem of machine unlearning by analyzing how Sharpness-Aware Minimization (SAM) affects the trade-off between retaining useful data and forgetting unwanted signals, showing that SAM outperforms SGD with relaxed requirements for retain signals and enhances unlearning methods, achieving decreased feature entanglement and stronger resistance to attacks.
We characterize the effectiveness of Sharpness-aware minimization (SAM) under machine unlearning scheme, where unlearning forget signals interferes with learning retain signals. While previous work prove that SAM improves generalization with noise memorization prevention, we show that SAM abandons such denoising property when fitting the forget set, leading to various test error bounds depending on signal strength. We further characterize the signal surplus of SAM in the order of signal strength, which enables learning from less retain signals to maintain model performance and putting more weight on unlearning the forget set. Empirical studies show that SAM outperforms SGD with relaxed requirement for retain signals and can enhance various unlearning methods either as pretrain or unlearn algorithm. Observing that overfitting can benefit more stringent sample-specific unlearning, we propose Sharp MinMax, which splits the model into two to learn retain signals with SAM and unlearn forget signals with sharpness maximization, achieving best performance. Extensive experiments show that SAM enhances unlearning across varying difficulties measured by data memorization, yielding decreased feature entanglement between retain and forget sets, stronger resistance to membership inference attacks, and a flatter loss landscape.