Cost-Matching Model Predictive Control for Efficient Reinforcement Learning in Humanoid Locomotion
This work addresses locomotion efficiency and robustness for humanoid robots, representing an incremental improvement over existing methods.
The paper tackles the problem of efficient reinforcement learning for humanoid locomotion by proposing a cost-matching approach within a Model Predictive Control framework, resulting in improved performance and robustness compared to manually tuned baselines in simulation.
In this paper, we propose a cost-matching approach for optimal humanoid locomotion within a Model Predictive Control (MPC)-based Reinforcement Learning (RL) framework. A parameterized MPC formulation with centroidal dynamics is trained to approximate the action-value function obtained from high-fidelity closed-loop data. Specifically, the MPC cost-to-go is evaluated along recorded state-action trajectories, and the parameters are updated to minimize the discrepancy between MPC-predicted values and measured returns. This formulation enables efficient gradient-based learning while avoiding the computational burden of repeatedly solving the MPC problem during training. The proposed method is validated in simulation using a commercial humanoid platform. Results demonstrate improved locomotion performance and robustness to model mismatch and external disturbances compared with manually tuned baselines.