KirchhoffNet: A Scalable Ultra Fast Analog Neural Network
This proposes a new paradigm for scalable, ultra-fast analog neural networks, potentially enabling low-power AI hardware implementations.
The paper introduces KirchhoffNet, an analog neural network based on Kirchhoff's laws that uses node voltages as inputs/outputs and learnable parameters on edges, achieving state-of-the-art performance on complex tasks without traditional layers. It can be implemented as a low-power analog circuit with forward calculation completed in a short time regardless of parameter count.
In this paper, we leverage a foundational principle of analog electronic circuitry, Kirchhoff's current and voltage laws, to introduce a distinctive class of neural network models termed KirchhoffNet. Essentially, KirchhoffNet is an analog circuit that can function as a neural network, utilizing its initial node voltages as the neural network input and the node voltages at a specific time point as the output. The evolution of node voltages within the specified time is dictated by learnable parameters on the edges connecting nodes. We demonstrate that KirchhoffNet is governed by a set of ordinary differential equations (ODEs), and notably, even in the absence of traditional layers (such as convolution layers), it attains state-of-the-art performances across diverse and complex machine learning tasks. Most importantly, KirchhoffNet can be potentially implemented as a low-power analog integrated circuit, leading to an appealing property -- irrespective of the number of parameters within a KirchhoffNet, its on-chip forward calculation can always be completed within a short time. This characteristic makes KirchhoffNet a promising and fundamental paradigm for implementing large-scale neural networks, opening a new avenue in analog neural networks for AI.