How to Train Your Resistive Network: Generalized Equilibrium Propagation and Analytical Learning
This addresses the challenge of energy-efficient analog hardware for machine learning, offering a method to train such systems locally, though it appears incremental as it builds on existing algorithms like Equilibrium Propagation.
The paper tackled the problem of training analog computing systems like resistive networks under physical locality constraints, by developing an algorithm to exactly calculate gradients using a graph theoretic framework and introducing Generalized Equilibrium Propagation, demonstrating through simulations that training can be done without replicas or full readouts and with partial updates.
Machine learning is a powerful method of extracting meaning from data; unfortunately, current digital hardware is extremely energy-intensive. There is interest in an alternative analog computing implementation that could match the performance of traditional machine learning while being significantly more energy-efficient. However, it remains unclear how to train such analog computing systems while adhering to locality constraints imposed by the physical (as opposed to digital) nature of these systems. Local learning algorithms such as Equilibrium Propagation and Coupled Learning have been proposed to address this issue. In this paper, we develop an algorithm to exactly calculate gradients using a graph theoretic and analytical framework for Kirchhoff's laws. We also introduce Generalized Equilibrium Propagation, a framework encompassing a broad class of Hebbian learning algorithms, including Coupled Learning and Equilibrium Propagation, and show how our algorithm compares. We demonstrate our algorithm using numerical simulations and show that we can train resistor networks without the need for a replica or readout over all resistors, only at the output layer. We also show that under the analytical gradient approach, it is possible to update only a subset of the resistance values without a strong degradation in performance.