Repairing Neural Networks by Leaving the Right Past Behind
This work addresses the issue of model failures due to data deficiencies for machine learning practitioners, offering a novel repair mechanism that builds on continual learning.
The paper tackles the problem of repairing neural network failures caused by deficient training data by developing a framework that identifies detrimental examples and erases their influence, resulting in improved performance over baselines for both identification and repair tasks.
Prediction failures of machine learning models often arise from deficiencies in training data, such as incorrect labels, outliers, and selection biases. However, such data points that are responsible for a given failure mode are generally not known a priori, let alone a mechanism for repairing the failure. This work draws on the Bayesian view of continual learning, and develops a generic framework for both, identifying training examples that have given rise to the target failure, and fixing the model through erasing information about them. This framework naturally allows leveraging recent advances in continual learning to this new problem of model repairment, while subsuming the existing works on influence functions and data deletion as specific instances. Experimentally, the proposed approach outperforms the baselines for both identification of detrimental training data and fixing model failures in a generalisable manner.