Equilibrium Propagation and Hamiltonian Inference in the Diffusive Fitzhugh-Nagumo Model
Provides a theoretical unification of two learning frameworks for a specific class of neural networks, but the practical impact is limited to the Fitzhugh-Nagumo model.
This work extends Equilibrium Propagation to skew-gradient systems, showing equivalence between deep Energy-Based Models and Hamiltonian neural networks, and derives a Hamiltonian recurrence for inference in Fitzhugh-Nagumo networks and deep Energy-Based Models.
In this work, we extend the Equilibrium Propagation framework to skew-gradient systems and show an equivalence between deep Energy-Based Models and Hamiltonian neural networks. We focus on networks of diffusively coupled Fitzhugh-Nagumo neurons as a prototypical example. We show that since stationary solutions of the Fitzhugh-Nagumo model are described by self-adjoint operators, the methods of equilibrium propagation for performing credit assignment can be applied. Furthermore, for Fitzhugh-Nagumo networks with the topology of a deep residual network, we show that the steady state solutions admit a (spatial) Hamiltonian, and thus the methods of Hamiltonian Echo Backpropagation can be applied. We end by deriving an explicit layer-wise Hamiltonian recurrence relation governing inference for stationary solutions of both deep Fitzhugh-Nagumo networks and deep Energy-Based Models.