Zhuangyu Han

Yusuf Ahmed Khan, Zhuangyu Han, Abhronil Sengupta

Memristive crossbar arrays have emerged as a promising platform for efficient on-chip learning, enabling local learning rules such as Equilibrium Propagation (EP) to be realized without the memory overhead of conventional backpropagation. However, as these devices age, permanent stuck-at (SA) faults accumulate at individual synaptic elements, irreversibly corrupting the stored weights and degrading model performance in ways that in-situ retraining alone cannot address. Existing mitigation strategies either rely on hardware redundancy at significant area and power cost, or require explicit fault localization that is impractical to perform continuously on-chip. This paper adopts a brain-inspired fault recovery route motivated by self-repair functionalities enabled by astrocytes -- a type of glial cell. We couple a computational neuroscience study of astrocytic neuromodulation under permanent synaptic faults with an algorithmic recovery mechanism for faulted crossbar-mapped networks. Our computational neuroscience study characterizes how astrocytes modulate surviving synapses in a bidirectional EP-specific network setting under both SA-0 and high-conductance stuck-at fault conditions, revealing that recovery operates at the neural population level. Motivated by this observation, we propose a retraining-based repair mechanism that augments EP with an additional repair nudge anchored to pre-fault activation targets of the healthy network, encouraging surviving weights to collectively reconstruct pre-fault internal representations without any information of which weights are faulty. To our knowledge, STAR is the first astrocyte-inspired repair mechanism to operate under a supervised local learning rule and to scale to convolutional networks.

Zhuangyu Han, Abhronil Sengupta

Reinforcement learning (RL) has enabled robust quadruped locomotion over complex terrain, but most learned controllers are trained offline with backpropagation in massively parallel simulation and deployed as fixed policies, limiting adaptation to terrain variation, payload changes, actuator wear, and other real-world conditions under onboard power constraints. Local learning provides a potential path toward energy-aware on-robot adaptation by replacing global backpropagation graphs with updates driven by local neural states, making the learning rule more compatible with neuromorphic and in-memory computing substrates. This work proposes an equilibrium-propagation (EP)-based proximal policy optimization (PPO) framework for uneven-terrain quadruped locomotion. The controller combines a bio-inspired central pattern generator (CPG) policy with a residual postural adjustment policy, while replacing conventional backpropagation-trained policy and value networks with EP-enabled local learning. To train stochastic continuous-control policies with EP, we derive an EP-compatible PPO output-nudging signal and introduce a two-sided ratio clipping mechanism that stabilizes policy updates during relaxation. Experiments on a 12-DoF A1 quadruped show that the proposed controller achieves stable policy convergence in a two-stage uneven terrain locomotion task. Its locomotion performance is comparable to a backpropagation-trained PPO baseline in success rate, velocity tracking, actuator power, and body stability, while improving GPU memory efficiency by 4.3\(\times\) compared with backpropagation through time (BPTT). These results suggest that local equilibrium-based learning can support high-dimensional embodied locomotion and provide an algorithmic foundation for low-power on-robot adaptation and fine-tuning.