Optimal Control of Agent-Based Dynamics under Deep Galerkin Feedback Laws
This work addresses sampling challenges in high-dimensional control for agent-based dynamics, offering incremental improvements in policy approximation for specific models like opinion dynamics.
The paper tackles high-dimensional control problems by addressing sampling issues in the Deep Galerkin Method, proposing a drift relaxation-based sampling approach that reduces variance in policy approximations. This method is validated on mean-field control problems, showing significant cost reductions over manually optimized controls and improvements over the Deep FBSDE approach on the Linear-Quadratic Regulator problem.
Ever since the concepts of dynamic programming were introduced, one of the most difficult challenges has been to adequately address high-dimensional control problems. With growing dimensionality, the utilisation of Deep Neural Networks promises to circumvent the issue of an otherwise exponentially increasing complexity. The paper specifically investigates the sampling issues the Deep Galerkin Method is subjected to. It proposes a drift relaxation-based sampling approach to alleviate the symptoms of high-variance policy approximations. This is validated on mean-field control problems; namely, the variations of the opinion dynamics presented by the Sznajd and the Hegselmann-Krause model. The resulting policies induce a significant cost reduction over manually optimised control functions and show improvements on the Linear-Quadratic Regulator problem over the Deep FBSDE approach.