Are skip connections necessary for biologically plausible learning rules?
This addresses the challenge of developing competitive biologically inspired alternatives to backpropagation for machine learning researchers, though it is incremental as it builds on existing rules.
The paper tackled the problem of biologically plausible learning rules underperforming compared to backpropagation, showing that incorporating skip connections enables these rules to match backpropagation's performance on MNIST with robust hyper-parameter handling.
Backpropagation is the workhorse of deep learning, however, several other biologically-motivated learning rules have been introduced, such as random feedback alignment and difference target propagation. None of these methods have produced a competitive performance against backpropagation. In this paper, we show that biologically-motivated learning rules with skip connections between intermediate layers can perform as well as backpropagation on the MNIST dataset and are robust to various sets of hyper-parameters.