Soft Switching Expert Policies for Controlling Systems with Uncertain Parameters
Addresses the reality gap in sim-to-real transfer for control systems with uncertain parameters, offering a practical method for robotics and autonomous systems.
Proposed a two-stage RL algorithm that learns multiple control policies in simulation for different system parameters, then adaptively switches them online using convex optimization, reducing learning complexity compared to single-policy approaches.
This paper proposes a simulation-based reinforcement learning algorithm for controlling systems with uncertain and varying system parameters. While simulators are useful for safely learning control policies, the reality gap remains a major challenge. To alleviate this challenge, we propose a two-stage algorithm. First, multiple control policies are learned for systems with different system parameters in a simulator. Second, for a real system, the control policies are adaptively switched using an online convex optimization algorithm based on observations. This approach is expected to reduce learning complexity compared with existing approaches that rely on a single policy to address the reality gap.