SineProject: Machine Unlearning for Stable Vision Language Alignment
This addresses the need for safe and private knowledge removal in MLLMs, representing an incremental improvement over prior unlearning methods.
The paper tackled the problem of machine unlearning in multimodal large language models, where existing methods disrupt vision-language alignment, and introduced SineProject, which reduces benign query refusals by 100% while achieving complete forgetting of targeted information with negligible computational overhead.
Multimodal Large Language Models (MLLMs) increasingly need to forget specific knowledge such as unsafe or private information without requiring full retraining. However, existing unlearning methods often disrupt vision language alignment, causing models to reject both harmful and benign queries. We trace this failure to the projector network during unlearning, its Jacobian becomes severely illconditioned, leading to unstable optimization and drift in cross modal embeddings. We introduce SineProject, a simple method that augments the frozen projector with sinusoidally modulated trainable parameters, improving the Jacobian's spectral conditioning and stabilizing alignment throughout unlearning. Across standard safety and privacy unlearning benchmarks using LLaVA v1.5 7B and 13B, SineProject reduces benign query refusals while achieving complete forgetting of targeted information, yielding state of the art forget retain trade offs with negligible computational overhead.