Multi-Objective Reference-Aligned Machine Unlearning
For machine learning practitioners needing to comply with data deletion requests, RAUL provides a more effective unlearning method that minimizes model degradation.
Machine unlearning aims to remove specific training data influence while preserving model utility. Existing single-objective methods cause catastrophic forgetting; the proposed RAUL framework uses multi-objective optimization with bounded KL alignment to achieve unlearning performance closest to full retraining.
Machine unlearning aims to remove the influence of specific training samples while preserving the model's utility. Existing single-objective approaches, such as gradient ascent or random relabeling, often induce catastrophic forgetting due to conflicting optimization dynamics and unbounded forgetting objectives that cause the model to drift from its pre-trained knowledge. We propose Reference-Aligned UnLearning (RAUL), a multi-objective framework that jointly optimizes forgetting and retention by replacing unbounded loss maximization with a bounded KL alignment of predictions on forgotten samples toward a reference distribution representing unseen data, instantiated either as a uniform distribution or an empirical distribution from a held-out reference set, which constrains the forgetting objective and reduces gradient conflict with retention. The resulting multi-objective optimization (MOO) problem is solved via Jacobian descent, which aggregates multiple gradients into a direction that does not conflict. Our results demonstrate that RAUL achieves the closest gap compared to full retraining.