4.4NEDec 4, 2019
Are skip connections necessary for biologically plausible learning rules?Daniel Jiwoong Im, Rutuja Patil, Kristin Branson
Backpropagation is the workhorse of deep learning, however, several other biologically-motivated learning rules have been introduced, such as random feedback alignment and difference target propagation. None of these methods have produced a competitive performance against backpropagation. In this paper, we show that biologically-motivated learning rules with skip connections between intermediate layers can perform as well as backpropagation on the MNIST dataset and are robust to various sets of hyper-parameters.