Predictive Coding, Variational Autoencoders, and Biological Connections
It aims to bridge neuroscience and machine learning by highlighting theoretical connections, but is incremental as it reviews and suggests ideas rather than presenting new empirical results.
This paper reviews predictive coding from neuroscience and variational autoencoders from machine learning, identifying their common mathematical framework to enhance dialogue between the fields, and proposes correspondences such as cortical dendrites as deep networks and lateral inhibition as normalizing flows.
This paper reviews predictive coding, from theoretical neuroscience, and variational autoencoders, from machine learning, identifying the common origin and mathematical framework underlying both areas. As each area is prominent within its respective field, more firmly connecting these areas could prove useful in the dialogue between neuroscience and machine learning. After reviewing each area, we discuss two possible correspondences implied by this perspective: cortical pyramidal dendrites as analogous to (non-linear) deep networks and lateral inhibition as analogous to normalizing flows. These connections may provide new directions for further investigations in each field.