Minh Nghia Le

Minh Le, Subhradeep Kayal

The ability to detect edges is a fundamental attribute necessary to truly capture visual concepts. In this paper, we prove that edges cannot be represented properly in the first convolutional layer of a neural network, and further show that they are poorly captured in popular neural network architectures such as VGG-16 and ResNet. The neural networks are found to rely on color information, which might vary in unexpected ways outside of the datasets used for their evaluation. To improve their robustness, we propose edge-detection units and show that they reduce performance loss and generate qualitatively different representations. By comparing various models, we show that the robustness of edge detection is an important factor contributing to the robustness of models against color noise.

Minh Le

Deep neural networks have achieved impressive performance and become the de-facto standard in many tasks. However, troubling phenomena such as adversarial and fooling examples suggest that the generalization they make is flawed. I argue that among the roots of the phenomena are two geometric properties of common deep learning architectures: their distributed nature and the connectedness of their decision regions. As a remedy, I propose new architectures inspired by fuzzy logic that combine several alternative design elements. Through experiments on MNIST and CIFAR-10, the new models are shown to be more local, better at rejecting noise samples, and more robust against adversarial examples. Ablation analyses reveal behaviors on adversarial examples that cannot be explained by the linearity hypothesis but are consistent with the hypothesis that logic-inspired traits create more robust models.