Learning to Play Soccer by Reinforcement and Applying Sim-to-Real to Compete in the Real World
This is an incremental application of RL and sim-to-real methods to a specific robotics domain, potentially improving autonomous robot performance in soccer competitions.
The authors tackled the problem of controlling soccer robots in the IEEE Very Small Size Soccer league using reinforcement learning, achieving a policy that beat a human-designed policy from the third-place team in 1-vs-1 matches.
This work presents an application of Reinforcement Learning (RL) for the complete control of real soccer robots of the IEEE Very Small Size Soccer (VSSS), a traditional league in the Latin American Robotics Competition (LARC). In the VSSS league, two teams of three small robots play against each other. We propose a simulated environment in which continuous or discrete control policies can be trained, and a Sim-to-Real method to allow using the obtained policies to control a robot in the real world. The results show that the learned policies display a broad repertoire of behaviors that are difficult to specify by hand. This approach, called VSSS-RL, was able to beat the human-designed policy for the striker of the team ranked 3rd place in the 2018 LARC, in 1-vs-1 matches.