Optimal Control of Complex Systems through Variational Inference with a Discrete Event Decision Process
This addresses the problem of controlling high-dimensional, non-linear systems like traffic management for practitioners, though it appears incremental as it builds on existing simulation and analytical approaches.
The paper tackles optimal control of complex social systems by formulating the problem as a discrete event decision process and reducing it to variational inference with Bethe entropy approximation, achieving higher expected rewards, faster convergence, and lower variance in a real-world transportation scenario compared to state-of-the-art methods.
Complex social systems are composed of interconnected individuals whose interactions result in group behaviors. Optimal control of a real-world complex system has many applications, including road traffic management, epidemic prevention, and information dissemination. However, such real-world complex system control is difficult to achieve because of high-dimensional and non-linear system dynamics, and the exploding state and action spaces for the decision maker. Prior methods can be divided into two categories: simulation-based and analytical approaches. Existing simulation approaches have high-variance in Monte Carlo integration, and the analytical approaches suffer from modeling inaccuracy. We adopted simulation modeling in specifying the complex dynamics of a complex system, and developed analytical solutions for searching optimal strategies in a complex network with high-dimensional state-action space. To capture the complex system dynamics, we formulate the complex social network decision making problem as a discrete event decision process. To address the curse of dimensionality and search in high-dimensional state action spaces in complex systems, we reduce control of a complex system to variational inference and parameter learning, introduce Bethe entropy approximation, and develop an expectation propagation algorithm. Our proposed algorithm leads to higher system expected rewards, faster convergence, and lower variance of value function in a real-world transportation scenario than state-of-the-art analytical and sampling approaches.