Gibbs Sampling with Low-Power Spiking Digital Neurons
This addresses the problem of energy-efficient hardware implementation for machine learning algorithms, but it is incremental as it applies an existing method to new hardware.
The paper implemented Gibbs sampling for Restricted Boltzmann Machines and Deep Belief Networks using low-power digital spiking neurons with stochastic properties, demonstrating performance metrics for inference tasks in generative and discriminative contexts.
Restricted Boltzmann Machines and Deep Belief Networks have been successfully used in a wide variety of applications including image classification and speech recognition. Inference and learning in these algorithms uses a Markov Chain Monte Carlo procedure called Gibbs sampling. A sigmoidal function forms the kernel of this sampler which can be realized from the firing statistics of noisy integrate-and-fire neurons on a neuromorphic VLSI substrate. This paper demonstrates such an implementation on an array of digital spiking neurons with stochastic leak and threshold properties for inference tasks and presents some key performance metrics for such a hardware-based sampler in both the generative and discriminative contexts.