Variational Probability Flow for Biologically Plausible Training of Deep Neural Networks
This work addresses the need for more efficient and biologically interpretable training algorithms in deep learning, offering a novel approach that could bridge artificial and biological neural networks, though it is incremental in the context of existing biologically plausible methods.
The paper tackled the problem of training deep neural networks with biologically plausible methods by proposing Variational Probability Flow (VPF), an algorithm for training binary Deep Boltzmann Machines that achieves local weight updates without Gibbs sampling or backpropagation, resulting in accurate image reconstruction and high log-likelihood on MNIST and Fashion MNIST datasets.
The quest for biologically plausible deep learning is driven, not just by the desire to explain experimentally-observed properties of biological neural networks, but also by the hope of discovering more efficient methods for training artificial networks. In this paper, we propose a new algorithm named Variational Probably Flow (VPF), an extension of minimum probability flow for training binary Deep Boltzmann Machines (DBMs). We show that weight updates in VPF are local, depending only on the states and firing rates of the adjacent neurons. Unlike contrastive divergence, there is no need for Gibbs confabulations; and unlike backpropagation, alternating feedforward and feedback phases are not required. Moreover, the learning algorithm is effective for training DBMs with intra-layer connections between the hidden nodes. Experiments with MNIST and Fashion MNIST demonstrate that VPF learns reasonable features quickly, reconstructs corrupted images more accurately, and generates samples with a high estimated log-likelihood. Lastly, we note that, interestingly, if an asymmetric version of VPF exists, the weight updates directly explain experimental results in Spike-Timing-Dependent Plasticity (STDP).