Deep Networks with Internal Selective Attention through Feedback Connections
This addresses the limitation of traditional CNNs for computer vision tasks by enabling dynamic internal attention, though it appears incremental as it builds on existing feedback and attention concepts.
The paper tackled the problem of stationary and feedforward convolutional neural networks by introducing a Deep Attention Selective Network (dasNet) architecture with feedback connections, which dynamically alters convolutional filter sensitivities during classification, resulting in outperforming the previous state-of-the-art on CIFAR-10 and CIFAR-100 datasets.
Traditional convolutional neural networks (CNN) are stationary and feedforward. They neither change their parameters during evaluation nor use feedback from higher to lower layers. Real brains, however, do. So does our Deep Attention Selective Network (dasNet) architecture. DasNets feedback structure can dynamically alter its convolutional filter sensitivities during classification. It harnesses the power of sequential processing to improve classification performance, by allowing the network to iteratively focus its internal attention on some of its convolutional filters. Feedback is trained through direct policy search in a huge million-dimensional parameter space, through scalable natural evolution strategies (SNES). On the CIFAR-10 and CIFAR-100 datasets, dasNet outperforms the previous state-of-the-art model.