Robust Learning via Ensemble Density Propagation in Deep Neural Networks
This addresses the problem of making deep neural networks more reliable for applications in noisy or adversarial environments, representing a novel method for a known bottleneck.
The paper tackled robust learning in deep neural networks under uncertain or adversarial conditions by proposing Ensemble Density Propagation (EnDP), which propagates moments of variational distributions to estimate predictive outputs, resulting in significant improvements in robustness to noise and attacks on MNIST and CIFAR-10 datasets.
Learning in uncertain, noisy, or adversarial environments is a challenging task for deep neural networks (DNNs). We propose a new theoretically grounded and efficient approach for robust learning that builds upon Bayesian estimation and Variational Inference. We formulate the problem of density propagation through layers of a DNN and solve it using an Ensemble Density Propagation (EnDP) scheme. The EnDP approach allows us to propagate moments of the variational probability distribution across the layers of a Bayesian DNN, enabling the estimation of the mean and covariance of the predictive distribution at the output of the model. Our experiments using MNIST and CIFAR-10 datasets show a significant improvement in the robustness of the trained models to random noise and adversarial attacks.