Beyond Uniform Deletion: A Data Value-Weighted Framework for Certified Machine Unlearning
This work addresses the need for efficient and privacy-preserving model updates in machine learning, particularly for legislated rights like the right to be forgotten, though it is incremental by building on existing unlearning methods.
The paper tackles the problem of machine unlearning by proposing a data value-weighted framework (DVWU) that accounts for heterogeneous data contributions, resulting in superior predictive performance and robustness compared to conventional methods, as demonstrated on synthetic and real-world datasets.
As the right to be forgotten becomes legislated worldwide, machine unlearning mechanisms have emerged to efficiently update models for data deletion and enhance user privacy protection. However, existing machine unlearning algorithms frequently neglect the fact that different data points may contribute unequally to model performance (i.e., heterogeneous data values). Treat them equally in machine unlearning procedure can potentially degrading the performance of updated models. To address this limitation, we propose Data Value-Weighted Unlearning (DVWU), a general unlearning framework that accounts for data value heterogeneity into the unlearning process. Specifically, we design a weighting strategy based on data values, which are then integrated into the unlearning procedure to enable differentiated unlearning for data points with varying utility to the model. The DVWU framework can be broadly adapted to various existing machine unlearning methods. We use the one-step Newton update as an example for implementation, developing both output and objective perturbation algorithms to achieve certified unlearning. Experiments on both synthetic and real-world datasets demonstrate that our methods achieve superior predictive performance and robustness compared to conventional unlearning approaches. We further show the extensibility of our framework on gradient ascent method by incorporating the proposed weighting strategy into the gradient terms, highlighting the adaptability of DVWU for broader gradient-based deep unlearning methods.