CLEAR: Character Unlearning in Textual and Visual Modalities
This addresses the problem of evaluating cross-modal data removal for researchers in machine unlearning, though it is incremental as it builds on existing unimodal methods.
The authors tackled the lack of benchmarks for multimodal unlearning (MMU) by introducing CLEAR, an open-source dataset with 200 fictitious individuals and 3,700 images, and showed that jointly unlearning both modalities outperforms single-modality approaches in evaluations of 11 methods.
Machine Unlearning (MU) is critical for removing private or hazardous information from deep learning models. While MU has advanced significantly in unimodal (text or vision) settings, multimodal unlearning (MMU) remains underexplored due to the lack of open benchmarks for evaluating cross-modal data removal. To address this gap, we introduce CLEAR, the first open-source benchmark designed specifically for MMU. CLEAR contains 200 fictitious individuals and 3,700 images linked with corresponding question-answer pairs, enabling a thorough evaluation across modalities. We conduct a comprehensive analysis of 11 MU methods (e.g., SCRUB, gradient ascent, DPO) across four evaluation sets, demonstrating that jointly unlearning both modalities outperforms single-modality approaches. The dataset is available at https://huggingface.co/datasets/therem/CLEAR