Rethinking Federated Unlearning via the Lens of Memorization
For federated learning systems needing to comply with privacy regulations, this work addresses ineffective unlearning and client unfairness caused by data overlap.
Federated unlearning methods often fail due to overlapping information between forgotten and retained data. The authors propose FedMemPrune, a pruning-based approach that removes unique memorized information, achieving unlearning performance close to retraining baselines while better eliminating memorization.
Federated learning (FL) increasingly needs machine unlearning to comply with privacy regulations. However, existing federated unlearning approaches may overlook the overlapping information between the unlearning and remaining data, leading to ineffective unlearning and unfairness between clients. In this work, we revisit federated unlearning through the lens of memorization. We argue that unlearning should mainly remove the unique memorized information attributable to the data to be forgotten, while preserving overlapping patterns that are also supported by the remaining data. Specifically, we propose Grouped Memorization Evaluation, an example-level metric that separates memorized knowledge from overlapping knowledge. Building on this metric, we introduce Federated Memorization Pruning (FedMemPrune), a pruning-based unlearning approach that resets redundant parameters responsible for memorization. Extensive experiments show that FedMemPrune closely matches retraining-based unlearning baselines while more effectively eliminating memorization than existing federated unlearning algorithms, yielding strong unlearning performance without sacrificing the utility of retained knowledge.