Prospects for Analog Circuits in Deep Networks
This work addresses the power efficiency challenge in deep learning hardware for edge or tiny machine learning applications, but it is incremental as it builds on existing analog circuit reviews and emerging memory technologies.
The paper reviews analog circuit designs for implementing machine learning operations and discusses their potential for low-power deep network accelerators, highlighting the use of emerging non-volatile memory technologies to reduce power consumption dominated by off-chip DRAM access.
Operations typically used in machine learning al-gorithms (e.g. adds and soft max) can be implemented bycompact analog circuits. Analog Application-Specific Integrated Circuit (ASIC) designs that implement these algorithms using techniques such as charge sharing circuits and subthreshold transistors, achieve very high power efficiencies. With the recent advances in deep learning algorithms, focus has shifted to hardware digital accelerator designs that implement the prevalent matrix-vector multiplication operations. Power in these designs is usually dominated by the memory access power of off-chip DRAM needed for storing the network weights and activations. Emerging dense non-volatile memory technologies can help to provide on-chip memory and analog circuits can be well suited to implement the needed multiplication-vector operations coupled with in-computing memory approaches. This paper presents abrief review of analog designs that implement various machine learning algorithms. It then presents an outlook for the use ofanalog circuits in low-power deep network accelerators suitable for edge or tiny machine learning applications.