Monolithic vs. hybrid controller for multi-objective Sim-to-Real learning
This addresses the challenge of applying simulation-to-real learning to complex multi-objective robotic tasks, though it is incremental as it compares existing approaches rather than introducing a new method.
The paper tackled the problem of training controllers for robotic tasks with multiple simultaneous objectives, such as reaching a target while avoiding obstacles, using simulation-to-real learning. The result showed that a hybrid controller based on pre-trained single-objective controllers with switching was easier to train and achieved a better success-failure trade-off than a monolithic controller with a multi-term reward function, with verification in a real setup.
Simulation to real (Sim-to-Real) is an attractive approach to construct controllers for robotic tasks that are easier to simulate than to analytically solve. Working Sim-to-Real solutions have been demonstrated for tasks with a clear single objective such as "reach the target". Real world applications, however, often consist of multiple simultaneous objectives such as "reach the target" but "avoid obstacles". A straightforward solution in the context of reinforcement learning (RL) is to combine multiple objectives into a multi-term reward function and train a single monolithic controller. Recently, a hybrid solution based on pre-trained single objective controllers and a switching rule between them was proposed. In this work, we compare these two approaches in the multi-objective setting of a robot manipulator to reach a target while avoiding an obstacle. Our findings show that the training of a hybrid controller is easier and obtains a better success-failure trade-off than a monolithic controller. The controllers trained in simulator were verified by a real set-up.