Biological Plausibility and Representational Alignment of Feedback Alignment in Convolutional Networks
For researchers in biologically plausible learning, this work shows that modified FA can achieve representational alignment with BP in convolutional networks, but the results are incremental as they confirm existing hypotheses without major performance breakthroughs.
This paper evaluates five learning algorithms, including modified feedback alignment (FA) and backpropagation (BP), on convolutional networks with CIFAR-10, finding that modified FA algorithms converge to internal representations similar to BP, suggesting their success stems from mimicking BP's representational geometry.
The feedback alignment (FA) algorithm offers a biologically plausible alternative to backpropagation (BP) for training neural networks yet notably fails to scale to convolutional architectures. Modifications have been proposed to address this limitation, but at questionable cost to biological plausibility. In this paper, we evaluate five learning algorithms including modified FA and standard BP, applied to the same convolutional architecture with the CIFAR-10 dataset. We provide a tripartite comparative analysis focusing on biological plausibility, interpretability, and computational complexity. Our results indicate that modified FA algorithms converge on internal representations that are structurally similar to those produced by backpropagation. In particular, it appears the functional success of modified FA algorithms may be rooted in their ability to mimic the representational geometry of backpropagation, converging on similar representations despite relying on fundamentally different weight update mechanisms.