Equilibrium Propagation Without Limits
This enables learning with strong error signals in neural networks, addressing a bottleneck in biologically plausible learning algorithms.
The paper tackles the limitation of Equilibrium Propagation requiring infinitesimal perturbations by establishing a finite-nudge foundation for local credit assignment, proving that the gradient of Helmholtz free energy differences provides an exact gradient estimator without approximations or convexity constraints.
We liberate Equilibrium Propagation (EP) from the limit of infinitesimal perturbations by establishing a finite-nudge foundation for local credit assignment. By modeling network states as Gibbs-Boltzmann distributions rather than deterministic points, we prove that the gradient of the difference in Helmholtz free energy between a nudged and free phase is exactly the difference in expected local energy derivatives. This validates the classic Contrastive Hebbian Learning update as an exact gradient estimator for arbitrary finite nudging, requiring neither infinitesimal approximations nor convexity. Furthermore, we derive a generalized EP algorithm based on the path integral of loss-energy covariances, enabling learning with strong error signals that standard infinitesimal approximations cannot support.