Evaluating Machine Unlearning via Epistemic Uncertainty
This addresses the need for practical evaluation metrics in machine unlearning, relevant for compliance with privacy regulations like GDPR, but it is incremental as it focuses on improving assessment rather than unlearning itself.
The paper tackles the problem of evaluating machine unlearning algorithms, which remove specific data points from trained models, by proposing a new metric based on epistemic uncertainty, as current methods using adversarial attacks or retraining comparisons are deemed insufficient.
There has been a growing interest in Machine Unlearning recently, primarily due to legal requirements such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act. Thus, multiple approaches were presented to remove the influence of specific target data points from a trained model. However, when evaluating the success of unlearning, current approaches either use adversarial attacks or compare their results to the optimal solution, which usually incorporates retraining from scratch. We argue that both ways are insufficient in practice. In this work, we present an evaluation metric for Machine Unlearning algorithms based on epistemic uncertainty. This is the first definition of a general evaluation metric for Machine Unlearning to our best knowledge.