Gradient target propagation
This work proposes an alternative to backpropagation for training neural networks, but it appears incremental as it matches rather than surpasses existing methods.
The paper introduces a learning rule for neural networks that estimates target values for neurons to minimize a cost function, achieving results at least as good as backpropagation on MNIST, MNIST-Fashion, and CIFAR-10 datasets.
We report a learning rule for neural networks that computes how much each neuron should contribute to minimize a giving cost function via the estimation of its target value. By theoretical analysis, we show that this learning rule contains backpropagation, Hebian learning, and additional terms. We also give a general technique for weights initialization. Our results are at least as good as those obtained with backpropagation. The neural networks are trained and tested in three problems: MNIST, MNIST-Fashion, and CIFAR-10 datasets. The associated code is available at https://github.com/tiago939/target.