Preventing overfitting in deep learning using differential privacy
This addresses the problem of poor generalization in deep learning for practitioners with limited data, but appears incremental as it adapts an existing privacy technique.
The paper tackles overfitting in deep neural networks by applying a differential privacy-based approach to improve generalization, though no specific performance numbers are provided.
The use of Deep Neural Network based systems in the real world is growing. They have achieved state-of-the-art performance on many image, speech and text datasets. They have been shown to be powerful systems that are capable of learning detailed relationships and abstractions from the data. This is a double-edged sword which makes such systems vulnerable to learning the noise in the training set, thereby negatively impacting performance. This is also known as the problem of \emph{overfitting} or \emph{poor generalization}. In a practical setting, analysts typically have limited data to build models that must generalize to unseen data. In this work, we explore the use of a differential-privacy based approach to improve generalization in Deep Neural Networks.