Deviant Learning Algorithm: Learning Sparse Mismatch Representations through Time and Space
This work addresses performance and decision-making in neural systems, particularly for unsupervised learning, but appears incremental as it builds on existing predictive coding and Mismatch Negativity concepts.
The paper tackles the problem of improving neural learning systems by integrating predictive coding with Mismatch Negativity and a large number of synapses, proposing the Deviant Learning Algorithm, which achieves competitive predictions with very small datasets in numerical experiments.
Predictive coding (PDC) has recently attracted attention in the neuroscience and computing community as a candidate unifying paradigm for neuronal studies and artificial neural network implementations particularly targeted at unsupervised learning systems. The Mismatch Negativity (MMN) has also recently been studied in relation to PC and found to be a useful ingredient in neural predictive coding systems. Backed by the behavior of living organisms, such networks are particularly useful in forming spatio-temporal transitions and invariant representations of the input world. However, most neural systems still do not account for large number of synapses even though this has been shown by a few machine learning researchers as an effective and very important component of any neural system if such a system is to behave properly. Our major point here is that PDC systems with the MMN effect in addition to a large number of synapses can greatly improve any neural learning system's performance and ability to make decisions in the machine world. In this paper, we propose a novel bio-mimetic computational intelligence algorithm -- the Deviant Learning Algorithm, inspired by these key ideas and functional properties of recent brain-cognitive discoveries and theories. We also show by numerical experiments guided by theoretical insights, how our invented bio-mimetic algorithm can achieve competitive predictions even with very small problem specific data.