Neural Langevin Machine: a local asymmetric learning rule can be creative
This work addresses the need for interpretable and biologically plausible generative models in machine learning and neuroscience, though it appears incremental as it builds on existing concepts like Boltzmann-Gibbs measures.
The paper tackles the problem of generating and storing information in recurrent neural networks by introducing the neural Langevin machine, a generative model that uses neural Langevin dynamics for sampling and learning real datasets, achieving interpretability and simple training with a biologically plausible local asymmetric learning rule.
Fixed points of recurrent neural networks can be leveraged to store and generate information. These fixed points can be captured by the Boltzmann-Gibbs measure, which leads to neural Langevin dynamics that can be used for sampling and learning a real dataset. We call this type of generative model neural Langevin machine, which is interpretable due to its analytic form of distribution and is simple to train. Moreover, the learning process is derived as a local asymmetric plasticity rule, bearing biological relevance. Therefore, one can realize a continuous sampling of creative dynamics in a neural network, mimicking an imagination process in brain circuits. This neural Langevin machine may be another promising generative model, at least in its strength in circuit-based sampling and biologically plausible learning rule.