Towards an Adaptable and Generalizable Optimization Engine in Decision and Control: A Meta Reinforcement Learning Approach
This work addresses the need for adaptable and generalizable optimization engines in decision and control for sequential problems in non-stationary environments, representing an incremental improvement over existing methods.
The paper tackles the problem of improving sampling-based model predictive control (MPC) in non-stationary environments by proposing a meta-reinforcement learning approach to learn an optimizer that updates MPC controllers without expert demonstrations, enabling fast adaptation (e.g., few-shots) to unseen control tasks, with experimental results validating its effectiveness.
Sampling-based model predictive control (MPC) has found significant success in optimal control problems with non-smooth system dynamics and cost function. Many machine learning-based works proposed to improve MPC by a) learning or fine-tuning the dynamics/ cost function, or b) learning to optimize for the update of the MPC controllers. For the latter, imitation learning-based optimizers are trained to update the MPC controller by mimicking the expert demonstrations, which, however, are expensive or even unavailable. More significantly, many sequential decision-making problems are in non-stationary environments, requiring that an optimizer should be adaptable and generalizable to update the MPC controller for solving different tasks. To address those issues, we propose to learn an optimizer based on meta-reinforcement learning (RL) to update the controllers. This optimizer does not need expert demonstration and can enable fast adaptation (e.g., few-shots) when it is deployed in unseen control tasks. Experimental results validate the effectiveness of the learned optimizer regarding fast adaptation.