Francesco Romanò

Mohak Bhardwaj, Thomas Lampe, Michael Neunert et al. · deepmind

Recent advances in real-world applications of reinforcement learning (RL) have relied on the ability to accurately simulate systems at scale. However, domains such as fluid dynamical systems exhibit complex dynamic phenomena that are hard to simulate at high integration rates, limiting the direct application of modern deep RL algorithms to often expensive or safety critical hardware. In this work, we introduce "Box o Flows", a novel benchtop experimental control system for systematically evaluating RL algorithms in dynamic real-world scenarios. We describe the key components of the Box o Flows, and through a series of experiments demonstrate how state-of-the-art model-free RL algorithms can synthesize a variety of complex behaviors via simple reward specifications. Furthermore, we explore the role of offline RL in data-efficient hypothesis testing by reusing past experiences. We believe that the insights gained from this preliminary study and the availability of systems like the Box o Flows support the way forward for developing systematic RL algorithms that can be generally applied to complex, dynamical systems. Supplementary material and videos of experiments are available at https://sites.google.com/view/box-o-flows/home.

Stefano Dafarra, Gabriele Nava, Marie Charbonneau et al.

A common approach to the generation of walking patterns for humanoid robots consists in adopting a layered control architecture. This paper proposes an architecture composed of three nested control loops. The outer loop exploits a robot kinematic model to plan the footstep positions. In the mid layer, a predictive controller generates a Center of Mass trajectory according to the well-known table-cart model. Through a whole-body inverse kinematics algorithm, we can define joint references for position controlled walking. The outcomes of these two loops are then interpreted as inputs of a stack-of-task QP-based torque controller, which represents the inner loop of the presented control architecture. This resulting architecture allows the robot to walk also in torque control, guaranteeing higher level of compliance. Real world experiments have been carried on the humanoid robot iCub.

Stefano Dafarra, Francesco Romano, Gabriele Nava et al.

When balancing, a humanoid robot can be easily subjected to unexpected disturbances like external pushes. In these circumstances, reactive movements as steps become a necessary requirement in order to avoid potentially harmful falling states. In this paper we conceive a Model Predictive Controller which determines a desired set of contact wrenches by predicting the future evolution of the robot, while taking into account constraints switching in case of steps. The control inputs computed by this strategy, namely the desired contact wrenches, are directly obtained on the robot through a modification of the momentum-based whole-body torque controller currently implemented on iCub. The proposed approach is validated through simulations in a stepping scenario, revealing high robustness and reliability when executing a recovery strategy.

Stefano Dafarra, Francesco Romano, Francesco Nori

One of the challenges for the robotics community is to deploy robots which can reliably operate in real world scenarios together with humans. A crucial requirement for legged robots is the capability to properly balance on their feet, rejecting external disturbances. iCub is a state-of-the-art humanoid robot which has only recently started to balance on its feet. While the current balancing controller has proved successful in various scenarios, it still misses the capability to properly react to strong pushes by taking steps. This paper goes in this direction. It proposes and implements a control strategy based on the Capture Point concept [1]. Instead of relying on position control, like most of Capture Point related approaches, the proposed strategy generates references for the momentum-based torque controller already implemented on the iCub, thus extending its capabilities to react to external disturbances, while retaining the advantages of torque control when interacting with the environment. Experiments in the Gazebo simulator and on the iCub humanoid robot validate the proposed strategy.

Francesco Romano, Silvio Traversaro, Daniele Pucci et al.

In this paper, we propose a software abstraction layer to simplify the design and synthesis of whole-body controllers without making any preliminary assumptions on the control law to be implemented. The main advantage of the proposed library is the decoupling of the control software from implementation details, which are related to the robotic platform. Furthermore, the resulting code is more clean and concise than ad-hoc code, as it focuses only on the implementation of the control law. In addition, we present a reference implementation of the abstraction layer together with a Simulink interface to provide support to Model-Driven based development. We also show the implementation of a simple proportional-derivative plus gravity compensation control together with a more complex momentum-based bipedal balance controller.

Francesco Romano, Daniele Pucci, Silvio Traversaro et al.

Legged locomotion has received increasing attention from the robotics community. In this respect, contact stability plays a critical role in ensuring that robots maintain balance, and it is a key element for balancing and walking controllers. The Center of Pressure is a contact stability criterion that defines a point that must be kept strictly inside the support polygon in order to ensure postural stability. In this paper, we introduce the concept of the sensitivity of the static center of pressure: roughly speaking, the rate of change of the center of pressure with respect to the system equilibrium configurations. This new concept can be used as an additional criterion to assess the robustness of the contact stability. We show how the sensitivity of the center of pressure can also be used as a metric to assess balancing controllers by considering two state-of-the-art control strategies. The analytical analysis is performed on a simplified model, and validated during balancing tasks on the iCub humanoid robot.

Andrea Del Prete, Francesco Romano, Lorenzo Natale et al.

This paper presents a new technique to control highly redundant mechanical systems, such as humanoid robots. We take inspiration from two approaches. Prioritized control is a widespread multi-task technique in robotics and animation: tasks have strict priorities and they are satisfied only as long as they do not conflict with any higher-priority task. Optimal control instead formulates an optimization problem whose solution is either a feedback control policy or a feedforward trajectory of control inputs. We introduce strict priorities in multi-task optimal control problems, as an alternative to weighting task errors proportionally to their importance. This ensures the respect of the specified priorities, while avoiding numerical conditioning issues. We compared our approach with both prioritized control and optimal control with tests on a simulated robot with 11 degrees of freedom.