Learning Control Policies of Hodgkin-Huxley Neuronal Dynamics
This work addresses the challenge of personalized, real-time deep brain stimulation for patients with neurological disorders, representing an incremental improvement in control policy methods for this domain.
The paper tackled the problem of finding optimal neurostimulation strategies for deep brain stimulation by framing it as a control problem, using a neural network to approximate the value function offline for real-time control generation, and demonstrated accuracy for out-of-distribution samples and robustness to disturbances in numerical experiments.
We present a neural network approach for closed-loop deep brain stimulation (DBS). We cast the problem of finding an optimal neurostimulation strategy as a control problem. In this setting, control policies aim to optimize therapeutic outcomes by tailoring the parameters of a DBS system, typically via electrical stimulation, in real time based on the patient's ongoing neuronal activity. We approximate the value function offline using a neural network to enable generating controls (stimuli) in real time via the feedback form. The neuronal activity is characterized by a nonlinear, stiff system of differential equations as dictated by the Hodgkin-Huxley model. Our training process leverages the relationship between Pontryagin's maximum principle and Hamilton-Jacobi-Bellman equations to update the value function estimates simultaneously. Our numerical experiments illustrate the accuracy of our approach for out-of-distribution samples and the robustness to moderate shocks and disturbances in the system.