Including local feature interactions in deep non-negative matrix factorization networks improves performance
This work addresses a problem in neural network design for researchers, offering a biologically plausible method that is incremental in improving deep network performance.
The paper tackled the performance gap between deep convolutional networks with non-negative matrix factorization (NMF) modules and vanilla CNNs by adding local feature interactions, resulting in improved performance on benchmark data that exceeds that of vanilla CNNs of similar size.
The brain uses positive signals as a means of signaling. Forward interactions in the early visual cortex are also positive, realized by excitatory synapses. Only local interactions also include inhibition. Non-negative matrix factorization (NMF) captures the biological constraint of positive long-range interactions and can be implemented with stochastic spikes. While NMF can serve as an abstract formalization of early neural processing in the visual system, the performance of deep convolutional networks with NMF modules does not match that of CNNs of similar size. However, when the local NMF modules are each followed by a module that mixes the NMF's positive activities, the performances on the benchmark data exceed that of vanilla deep convolutional networks of similar size. This setting can be considered a biologically more plausible emulation of the processing in cortical (hyper-)columns with the potential to improve the performance of deep networks.