Differentially Private Continual Learning
This addresses privacy compliance for institutions like hospitals that need to delete old data, but it is incremental as it combines existing techniques in differential privacy and continual learning.
The paper tackles catastrophic forgetting in neural networks when historic data must be deleted for privacy, by introducing a differentially private continual learning framework that uses variational inference and private generative models to estimate past data likelihoods, achieving privacy-preserving retention of old lessons.
Catastrophic forgetting can be a significant problem for institutions that must delete historic data for privacy reasons. For example, hospitals might not be able to retain patient data permanently. But neural networks trained on recent data alone will tend to forget lessons learned on old data. We present a differentially private continual learning framework based on variational inference. We estimate the likelihood of past data given the current model using differentially private generative models of old datasets.