PACE: A Framework for Learning and Control in Linear Incomplete-Information Differential Games
This work addresses a challenge in multi-agent control and human-robot interaction by enabling agents to infer objectives in incomplete-information games, though it is incremental as it builds on existing differential game methods.
The paper tackles the problem of two-player linear quadratic differential games with incomplete information, where agents lack knowledge of each other's cost functions, by proposing the PACE framework for real-time cost parameter estimation and adaptive control, with theoretical guarantees for convergence and stability and numerical studies showing improved stability and convergence speed.
In this paper, we address the problem of a two-player linear quadratic differential game with incomplete information, a scenario commonly encountered in multi-agent control, human-robot interaction (HRI), and approximation methods for solving general-sum differential games. While solutions to such linear differential games are typically obtained through coupled Riccati equations, the complexity increases when agents have incomplete information, particularly when neither is aware of the other's cost function. To tackle this challenge, we propose a model-based Peer-Aware Cost Estimation (PACE) framework for learning the cost parameters of the other agent. In PACE, each agent treats its peer as a learning agent rather than a stationary optimal agent, models their learning dynamics, and leverages this dynamic to infer the cost function parameters of the other agent. This approach enables agents to infer each other's objective function in real time based solely on their previous state observations and dynamically adapt their control policies. Furthermore, we provide a theoretical guarantee for the convergence of parameter estimation and the stability of system states in PACE. Additionally, in our numerical studies, we demonstrate how modeling the learning dynamics of the other agent benefits PACE, compared to approaches that approximate the other agent as having complete information, particularly in terms of stability and convergence speed.