Bio-inspired Min-Nets Improve the Performance and Robustness of Deep Networks
This work addresses performance and robustness issues in deep learning for computer vision, but it is incremental as it builds on existing architectures like ResNet and DenseNet.
The authors tackled the problem of improving deep network performance and robustness by introducing Min-Nets, inspired by end-stopped cortical cells, which incorporate units that output the minimum of two learned filters; they showed that Min-Nets achieve better performance on Cifar-10 and increased robustness against JPEG compression artifacts.
Min-Nets are inspired by end-stopped cortical cells with units that output the minimum of two learned filters. We insert such Min-units into state-of-the-art deep networks, such as the popular ResNet and DenseNet, and show that the resulting Min-Nets perform better on the Cifar-10 benchmark. Moreover, we show that Min-Nets are more robust against JPEG compression artifacts. We argue that the minimum operation is the simplest way of implementing an AND operation on pairs of filters and that such AND operations introduce a bias that is appropriate given the statistics of natural images.