Dependence of Equilibrium Propagation Training Success on Network Architecture
This work addresses the challenge of implementing energy-efficient neuromorphic computing by providing guidelines for scaling up architectures in realistic experimental settings, though it is incremental in nature.
The study investigated how the performance of equilibrium propagation, a physics-based training method, is affected by using locally connected lattice architectures instead of dense networks, finding that sparse networks with local connections can achieve comparable performance.
The rapid rise of artificial intelligence has led to an unsustainable growth in energy consumption. This has motivated progress in neuromorphic computing and physics-based training of learning machines as alternatives to digital neural networks. Many theoretical studies focus on simple architectures like all-to-all or densely connected layered networks. However, these may be challenging to realize experimentally, e.g. due to connectivity constraints. In this work, we investigate the performance of the widespread physics-based training method of equilibrium propagation for more realistic architectural choices, specifically, locally connected lattices. We train an XY model and explore the influence of architecture on various benchmark tasks, tracking the evolution of spatially distributed responses and couplings during training. Our results show that sparse networks with only local connections can achieve performance comparable to dense networks. Our findings provide guidelines for further scaling up architectures based on equilibrium propagation in realistic settings.