Causal Fuzzing for Verifying Machine Unlearning
This work addresses the need for thorough testing to enhance adaptability, fairness, and privacy in ML models, though it appears incremental by building on causality for verification.
The paper tackles the problem of verifying machine unlearning in black-box models by proposing CAFÉ, a causality-based framework that detects residual influences missed by existing methods, achieving successful detection across five datasets and three model architectures while maintaining computational efficiency.
As machine learning models become increasingly embedded in decision-making systems, the ability to "unlearn" targeted data or features is crucial for enhancing model adaptability, fairness, and privacy in models which involves expensive training. To effectively guide machine unlearning, a thorough testing is essential. Existing methods for verification of machine unlearning provide limited insights, often failing in scenarios where the influence is indirect. In this work, we propose CAFÉ, a new causality based framework that unifies datapoint- and feature-level unlearning for verification of black-box ML models. CAFÉ evaluates both direct and indirect effects of unlearning targets through causal dependencies, providing actionable insights with fine-grained analysis. Our evaluation across five datasets and three model architectures demonstrates that CAFÉ successfully detects residual influence missed by baselines while maintaining computational efficiency.