Learning with hidden variables
This is an incremental review that synthesizes existing research to provide conceptual insights for understanding cortical learning and its relation to machine learning.
The paper reviews recent advancements in algorithms and learning rules for neural networks with hidden layers to learn features from sensory inputs like images, text, and audio, focusing on dynamical inputs and single neuron models.
Learning and inferring features that generate sensory input is a task continuously performed by cortex. In recent years, novel algorithms and learning rules have been proposed that allow neural network models to learn such features from natural images, written text, audio signals, etc. These networks usually involve deep architectures with many layers of hidden neurons. Here we review recent advancements in this area emphasizing, amongst other things, the processing of dynamical inputs by networks with hidden nodes and the role of single neuron models. These points and the questions they arise can provide conceptual advancements in understanding of learning in the cortex and the relationship between machine learning approaches to learning with hidden nodes and those in cortical circuits.