A Cognac Shot To Forget Bad Memories: Corrective Unlearning for Graph Neural Networks
This work addresses the issue of data manipulation propagation in GNNs for model developers, offering a practical solution for post-training correction, though it is incremental as it builds on the recently formulated problem of Corrective Unlearning.
The paper tackles the problem of removing adverse effects from manipulated or incorrect data in trained Graph Neural Networks (GNNs) by introducing a new method called Cognac, which recovers most performance of an oracle with fully corrected data while being 8x more efficient than retraining from scratch.
Graph Neural Networks (GNNs) are increasingly being used for a variety of ML applications on graph data. Because graph data does not follow the independently and identically distributed (i.i.d.) assumption, adversarial manipulations or incorrect data can propagate to other data points through message passing, which deteriorates the model's performance. To allow model developers to remove the adverse effects of manipulated entities from a trained GNN, we study the recently formulated problem of Corrective Unlearning. We find that current graph unlearning methods fail to unlearn the effect of manipulations even when the whole manipulated set is known. We introduce a new graph unlearning method, Cognac, which can unlearn the effect of the manipulation set even when only 5% of it is identified. It recovers most of the performance of a strong oracle with fully corrected training data, even beating retraining from scratch without the deletion set while being 8x more efficient. We hope our work assists GNN developers in mitigating harmful effects caused by issues in real-world data, post-training. Our code is publicly available at https://github.com/cognac-gnn-unlearning/corrective-unlearning-for-gnns