DualOptim: Enhancing Efficacy and Stability in Machine Unlearning with Dual Optimizers
This work addresses the practical deployment challenges of machine unlearning for researchers and practitioners, though it is incremental as it enhances existing algorithms rather than introducing a new paradigm.
The paper tackled the problem of machine unlearning methods being sensitive to hyperparameters and unstable in deployment by proposing DualOptim, which uses adaptive learning rates and decoupled momentum, resulting in significant improvements in efficacy and stability across tasks like image classification and large language models.
Existing machine unlearning (MU) approaches exhibit significant sensitivity to hyperparameters, requiring meticulous tuning that limits practical deployment. In this work, we first empirically demonstrate the instability and suboptimal performance of existing popular MU methods when deployed in different scenarios. To address this issue, we propose Dual Optimizer (DualOptim), which incorporates adaptive learning rate and decoupled momentum factors. Empirical and theoretical evidence demonstrates that DualOptim contributes to effective and stable unlearning. Through extensive experiments, we show that DualOptim can significantly boost MU efficacy and stability across diverse tasks, including image classification, image generation, and large language models, making it a versatile approach to empower existing MU algorithms.