Progressive Operational Perceptron with Memory
This work addresses the challenge of learning deeper architectures in neural networks for researchers in machine learning, though it appears incremental as it builds upon existing POP methods.
The authors tackled the problem of improving the learning capability of Progressive Operational Perceptrons (POPs) by incorporating a linear projection path with memory, which accelerates and augments progressive learning, leading to better data representations and surpassing the original POPs and related algorithms in experiments.
Generalized Operational Perceptron (GOP) was proposed to generalize the linear neuron model in the traditional Multilayer Perceptron (MLP) and this model can mimic the synaptic connections of the biological neurons that have nonlinear neurochemical behaviours. Progressive Operational Perceptron (POP) is a multilayer network composing of GOPs which is formed layer-wise progressively. In this work, we propose major modifications that can accelerate as well as augment the progressive learning procedure of POP by incorporating an information-preserving, linear projection path from the input to the output layer at each progressive step. The proposed extensions can be interpreted as a mechanism that provides direct information extracted from the previously learned layers to the network, hence the term "memory". This allows the network to learn deeper architectures with better data representations. An extensive set of experiments show that the proposed modifications can surpass the learning capability of the original POPs and other related algorithms.