Easy Data Unlearning Bench
This addresses the problem of complex and inconsistent evaluation setups for researchers in machine unlearning, though it is incremental as it builds on existing benchmarking efforts.
The authors tackled the challenge of evaluating machine unlearning methods by introducing a unified and extensible benchmarking suite that simplifies evaluation using the KLoM metric, providing precomputed resources and streamlined infrastructure for reproducible and fair comparisons.
Evaluating machine unlearning methods remains technically challenging, with recent benchmarks requiring complex setups and significant engineering overhead. We introduce a unified and extensible benchmarking suite that simplifies the evaluation of unlearning algorithms using the KLoM (KL divergence of Margins) metric. Our framework provides precomputed model ensembles, oracle outputs, and streamlined infrastructure for running evaluations out of the box. By standardizing setup and metrics, it enables reproducible, scalable, and fair comparison across unlearning methods. We aim for this benchmark to serve as a practical foundation for accelerating research and promoting best practices in machine unlearning. Our code and data are publicly available.