Meta-Learning Linear Quadratic Regulators: A Policy Gradient MAML Approach for Model-free LQR
This addresses efficient adaptation to unseen LQR tasks for control systems, but it is incremental as it builds on existing MAML methods.
The paper tackles learning linear quadratic regulators (LQR) in multi-task, model-free settings by applying policy gradient MAML, showing it produces stabilizing controllers close to optimal ones with a linear convergence rate in model-based scenarios, improving upon prior sub-linear rates.
We investigate the problem of learning linear quadratic regulators (LQR) in a multi-task, heterogeneous, and model-free setting. We characterize the stability and personalization guarantees of a policy gradient-based (PG) model-agnostic meta-learning (MAML) (Finn et al., 2017) approach for the LQR problem under different task-heterogeneity settings. We show that our MAML-LQR algorithm produces a stabilizing controller close to each task-specific optimal controller up to a task-heterogeneity bias in both model-based and model-free learning scenarios. Moreover, in the model-based setting, we show that such a controller is achieved with a linear convergence rate, which improves upon sub-linear rates from existing work. Our theoretical guarantees demonstrate that the learned controller can efficiently adapt to unseen LQR tasks.