POUR: A Provably Optimal Method for Unlearning Representations via Neural Collapse
This addresses the need for more effective unlearning in computer vision to remove specific concepts without retraining, though it is incremental as it builds on existing unlearning frameworks.
The paper tackles the problem of incomplete forgetting in machine unlearning by extending unlearning to the representation level, introducing POUR, a provably optimal method based on Neural Collapse theory that outperforms state-of-the-art methods on datasets like CIFAR-10/100 and PathMNIST.
In computer vision, machine unlearning aims to remove the influence of specific visual concepts or training images without retraining from scratch. Studies show that existing approaches often modify the classifier while leaving internal representations intact, resulting in incomplete forgetting. In this work, we extend the notion of unlearning to the representation level, deriving a three-term interplay between forgetting efficacy, retention fidelity, and class separation. Building on Neural Collapse theory, we show that the orthogonal projection of a simplex Equiangular Tight Frame (ETF) remains an ETF in a lower dimensional space, yielding a provably optimal forgetting operator. We further introduce the Representation Unlearning Score (RUS) to quantify representation-level forgetting and retention fidelity. Building on this, we introduce POUR (Provably Optimal Unlearning of Representations), a geometric projection method with closed-form (POUR-P) and a feature-level unlearning variant under a distillation scheme (POUR-D). Experiments on CIFAR-10/100 and PathMNIST demonstrate that POUR achieves effective unlearning while preserving retained knowledge, outperforming state-of-the-art unlearning methods on both classification-level and representation-level metrics.