DeltaGrad: Rapid retraining of machine learning models
This addresses the need for efficient model updates in applications like privacy and bias reduction, though it appears incremental as it builds on existing retraining approaches.
The paper tackles the problem of expensive retraining of machine learning models on slightly changed datasets, proposing the DeltaGrad algorithm that uses cached information to achieve rapid retraining with favorable comparisons to state-of-the-art methods.
Machine learning models are not static and may need to be retrained on slightly changed datasets, for instance, with the addition or deletion of a set of data points. This has many applications, including privacy, robustness, bias reduction, and uncertainty quantifcation. However, it is expensive to retrain models from scratch. To address this problem, we propose the DeltaGrad algorithm for rapid retraining machine learning models based on information cached during the training phase. We provide both theoretical and empirical support for the effectiveness of DeltaGrad, and show that it compares favorably to the state of the art.