Configurable p-Neurons Using Modular p-Bits
This work addresses hardware efficiency for probabilistic neural networks, offering incremental improvements in resource utilization for AI applications.
The paper tackled the limited variety of probabilistic activation functions in neural networks by introducing modular p-bits, which enable configurable p-neurons with functions like Logistic Sigmoid, Tanh, and ReLU, resulting in a 10x reduction in hardware resources compared to conventional implementations.
Probabilistic bits (p-bits) have recently been employed in neural networks (NNs) as stochastic neurons with sigmoidal probabilistic activation functions. Nonetheless, there remain a wealth of other probabilistic activation functions that are yet to be explored. Here we re-engineer the p-bit by decoupling its stochastic signal path from its input data path, giving rise to a modular p-bit that enables the realization of probabilistic neurons (p-neurons) with a range of configurable probabilistic activation functions, including a probabilistic version of the widely used Logistic Sigmoid, Tanh and Rectified Linear Unit (ReLU) activation functions. We present spintronic (CMOS + sMTJ) designs that show wide and tunable probabilistic ranges of operation. Finally, we experimentally implement digital-CMOS versions on an FPGA, with stochastic unit sharing, and demonstrate an order of magnitude (10x) saving in required hardware resources compared to conventional digital p-bit implementations.