Robust Deep Neural Networks Inspired by Fuzzy Logic
This addresses robustness issues in deep learning for AI safety, but it is incremental as it builds on existing logic-inspired ideas.
The paper tackles the problem of adversarial and fooling examples in deep neural networks by proposing new architectures inspired by fuzzy logic, which are shown to be more robust, with experiments on MNIST and CIFAR-10 demonstrating improved noise rejection and adversarial robustness.
Deep neural networks have achieved impressive performance and become the de-facto standard in many tasks. However, troubling phenomena such as adversarial and fooling examples suggest that the generalization they make is flawed. I argue that among the roots of the phenomena are two geometric properties of common deep learning architectures: their distributed nature and the connectedness of their decision regions. As a remedy, I propose new architectures inspired by fuzzy logic that combine several alternative design elements. Through experiments on MNIST and CIFAR-10, the new models are shown to be more local, better at rejecting noise samples, and more robust against adversarial examples. Ablation analyses reveal behaviors on adversarial examples that cannot be explained by the linearity hypothesis but are consistent with the hypothesis that logic-inspired traits create more robust models.