Risto Kojcev

Yue Leire Erro Nuin, Nestor Gonzalez Lopez, Elias Barba Moral et al.

We propose a novel framework for Deep Reinforcement Learning (DRL) in modular robotics to train a robot directly from joint states, using traditional robotic tools. We use an state-of-the-art implementation of the Proximal Policy Optimization, Trust Region Policy Optimization and Actor-Critic Kronecker-Factored Trust Region algorithms to learn policies in four different Modular Articulated Robotic Arm (MARA) environments. We support this process using a framework that communicates with typical tools used in robotics, such as Gazebo and Robot Operating System 2 (ROS 2). We evaluate several algorithms in modular robots with an empirical study in simulation.

Nestor Gonzalez Lopez, Yue Leire Erro Nuin, Elias Barba Moral et al.

This paper presents an upgraded, real world application oriented version of gym-gazebo, the Robot Operating System (ROS) and Gazebo based Reinforcement Learning (RL) toolkit, which complies with OpenAI Gym. The content discusses the new ROS 2 based software architecture and summarizes the results obtained using Proximal Policy Optimization (PPO). Ultimately, the output of this work presents a benchmarking system for robotics that allows different techniques and algorithms to be compared using the same virtual conditions. We have evaluated environments with different levels of complexity of the Modular Articulated Robotic Arm (MARA), reaching accuracies in the millimeter scale. The converged results show the feasibility and usefulness of the gym-gazebo 2 toolkit, its potential and applicability in industrial use cases, using modular robots.

Víctor Mayoral Vilches, Alejandro Hernández Cordero, Asier Bilbao Calvo et al.

Rather than programming, training allows robots to achieve behaviors that generalize better and are capable to respond to real-world needs. However, such training requires a big amount of experimentation which is not always feasible for a physical robot. In this work, we present robot_gym, a framework to accelerate robot training through simulation in the cloud that makes use of roboticists' tools, simplifying the development and deployment processes on real robots. We unveil that, for simple tasks, simple 3DoF robots require more than 140 attempts to learn. For more complex, 6DoF robots, the number of attempts increases to more than 900 for the same task. We demonstrate that our framework, for simple tasks, accelerates the robot training time by more than 33% while maintaining similar levels of accuracy and repeatability.

Risto Kojcev, Nora Etxezarreta, Alejandro Hernández et al.

We argue that hierarchical methods can become the key for modular robots achieving reconfigurability. We present a hierarchical approach for modular robots that allows a robot to simultaneously learn multiple tasks. Our evaluation results present an environment composed of two different modular robot configurations, namely 3 degrees-of-freedom (DoF) and 4DoF with two corresponding targets. During the training, we switch between configurations and targets aiming to evaluate the possibility of training a neural network that is able to select appropriate motor primitives and robot configuration to achieve the target. The trained neural network is then transferred and executed on a real robot with 3DoF and 4DoF configurations. We demonstrate how this technique generalizes to robots with different configurations and tasks.

Víctor Mayoral, Risto Kojcev, Nora Etxezarreta et al.

We argue that hardware modularity plays a key role in the convergence of Robotics and Artificial Intelligence (AI). We introduce a new approach for building robots that leads to more adaptable and capable machines. We present the concept of a self-adaptable robot that makes use of hardware modularity and AI techniques to reduce the effort and time required to be built. We demonstrate in simulation and with a real robot how, rather than programming, training produces behaviors in the robot that generalize fast and produce robust outputs in the presence of noise. In particular, we advocate for mammals.

Risto Kojcev, Nora Etxezarreta, Alejandro Hernández et al.

We propose a novel framework for Deep Reinforcement Learning (DRL) in modular robotics using traditional robotic tools that extend state-of-the-art DRL implementations and provide an end-to-end approach which trains a robot directly from joint states. Moreover, we present a novel technique to transfer these DLR methods into the real robot, aiming to close the simulation-reality gap. We demonstrate the robustness of the performance of state-of-the-art DRL methods for continuous action spaces in modular robots, with an empirical study both in simulation and in the real robot where we also evaluate how accelerating the simulation time affects the robot's performance. Our results show that extending the modular robot from 3 degrees-of-freedom (DoF), to 4 DoF, does not affect the robot's learning. This paves the way towards training modular robots using DRL techniques.

Irati Zamalloa, Iñigo Muguruza, Alejandro Hernández et al.

Today's landscape of robotics is dominated by vertical integration where single vendors develop the final product leading to slow progress, expensive products and customer lock-in. Opposite to this, an horizontal integration would result in a rapid development of cost-effective mass-market products with an additional consumer empowerment. The transition of an industry from vertical integration to horizontal integration is typically catalysed by de facto industry standards that enable a simplified and seamless integration of products. However, in robotics there is currently no leading candidate for a global plug-and-play standard. This paper tackles the problem of incompatibility between robot components that hinder the reconfigurability and flexibility demanded by the robotics industry. Particularly, it presents a model to create plug-and-play robot hardware components. Rather than iteratively evolving previous ontologies, our proposed model answers the needs identified by the industry while facilitating interoperability, measurability and comparability of robotics technology. Our approach differs significantly with the ones presented before as it is hardware-oriented and establishes a clear set of actions towards the integration of this model in real environments and with real manufacturers.

Irati Zamalloa, Risto Kojcev, Alejandro Hernández et al.

Robotics is called to be the next technological revolution and estimations indicate that it will trigger the fourth industrial revolution. This article presents a review of some of the most relevant milestones that occurred in robotics over the last few decades and future perspectives. Despite the fact that, nowadays, robotics is an emerging field, the challenges in many technological aspects and more importantly bringing innovative solutions to the market still remain open. The need of reducing the integration time, costs and a common hardware infrastructure are discussed and further analysed in this work. We conclude with a discussion of the future perspectives of robotics as an engineering discipline and with suggestions for future research directions.