ApproxDBN: Approximate Computing for Discriminative Deep Belief Networks
This work addresses power efficiency for hardware implementations of probabilistic generative neural networks, but it is incremental as it applies existing approximate computing techniques to a specific network type.
The paper tackled the challenge of high power consumption in hardware implementations of deep belief networks by developing ApproxDBN, a design methodology using approximate computing to reduce bit-lengths while maintaining target accuracy, achieving significant bit-length reduction with constrained accuracy loss.
Probabilistic generative neural networks are useful for many applications, such as image classification, speech recognition and occlusion removal. However, the power budget for hardware implementations of neural networks can be extremely tight. To address this challenge we describe a design methodology for using approximate computing methods to implement Approximate Deep Belief Networks (ApproxDBNs) by systematically exploring the use of (1) limited precision of variables; (2) criticality analysis to identify the nodes in the network which can operate with such limited precision while allowing the network to maintain target accuracy levels; and (3) a greedy search methodology with incremental retraining to determine the optimal reduction in precision to enable maximize power savings under user-specified accuracy constraints. Experimental results show that significant bit-length reduction can be achieved by our ApproxDBN with constrained accuracy loss.