Juan M. Gandarias

Luca Fortini, Mattia Leonori, Juan M. Gandarias et al.

Simulation tools are essential for robotics research, especially for those domains in which safety is crucial, such as Human-Robot Collaboration (HRC). However, it is challenging to simulate human behaviors, and existing robotics simulators do not integrate functional human models. This work presents Open-VICO, an open-source toolkit to integrate virtual human models in Gazebo focusing on vision-based human tracking. In particular, Open-VICO allows to combine in the same simulation environment realistic human kinematic models, multi-camera vision setups, and human-tracking techniques along with numerous robot and sensor models thanks to Gazebo. The possibility to incorporate pre-recorded human skeleton motion with Motion Capture systems broadens the landscape of human performance behavioral analysis within Human-Robot Interaction (HRI) settings. To describe the functionalities and stress the potential of the toolkit four specific examples, chosen among relevant literature challenges in the field, are developed using our simulation utils: i) 3D multi-RGB-D camera calibration in simulation, ii) creation of a synthetic human skeleton tracking dataset based on OpenPose, iii) multi-camera scenario for human skeleton tracking in simulation, and iv) a human-robot interaction example. The key of this work is to create a straightforward pipeline which we hope will motivate research on new vision-based algorithms and methodologies for lightweight human-tracking and flexible human-robot applications.

Francisco Pastor, Jorge García-González, Juan M. Gandarias et al.

Recent advances in the field of intelligent robotic manipulation pursue providing robotic hands with touch sensitivity. Haptic perception encompasses the sensing modalities encountered in the sense of touch (e.g., tactile and kinesthetic sensations). This letter focuses on multimodal object recognition and proposes analytical and data-driven methodologies to fuse tactile- and kinesthetic-based classification results. The procedure is as follows: a three-finger actuated gripper with an integrated high-resolution tactile sensor performs squeeze-and-release Exploratory Procedures (EPs). The tactile images and kinesthetic information acquired using angular sensors on the finger joints constitute the time-series datasets of interest. Each temporal dataset is fed to a Long Short-term Memory (LSTM) Neural Network, which is trained to classify in-hand objects. The LSTMs provide an estimation of the posterior probability of each object given the corresponding measurements, which after fusion allows to estimate the object through Bayesian and Neural inference approaches. An experiment with 36-classes is carried out to evaluate and compare the performance of the fused, tactile, and kinesthetic perception systems.The results show that the Bayesian-based classifiers improves capabilities for object recognition and outperforms the Neural-based approach.

Luca Fortini, Mattia Leonori, Juan M. Gandarias et al.

Tracking 3D human motion in real-time is crucial for numerous applications across many fields. Traditional approaches involve attaching artificial fiducial objects or sensors to the body, limiting their usability and comfort-of-use and consequently narrowing their application fields. Recent advances in Artificial Intelligence (AI) have allowed for markerless solutions. However, most of these methods operate in 2D, while those providing 3D solutions compromise accuracy and real-time performance. To address this challenge and unlock the potential of visual pose estimation methods in real-world scenarios, we propose a markerless framework that combines multi-camera views and 2D AI-based pose estimation methods to track 3D human motion. Our approach integrates a Weighted Least Square (WLS) algorithm that computes 3D human motion from multiple 2D pose estimations provided by an AI-driven method. The method is integrated within the Open-VICO framework allowing simulation and real-world execution. Several experiments have been conducted, which have shown high accuracy and real-time performance, demonstrating the high level of readiness for real-world applications and the potential to revolutionize human motion capture.

Juan M. Gandarias, Alfonso J. García-Cerezo, Jesús M. Gómez-de-Gabriel

Novel high-resolution pressure-sensor arrays allow treating pressure readings as standard images. Computer vision algorithms and methods such as Convolutional Neural Networks (CNN) can be used to identify contact objects. In this paper, a high-resolution tactile sensor has been attached to a robotic end-effector to identify contacted objects. Two CNN-based approaches have been employed to classify pressure images. These methods include a transfer learning approach using a pre-trained CNN on an RGB-images dataset and a custom-made CNN (TactNet) trained from scratch with tactile information. The transfer learning approach can be carried out by retraining the classification layers of the network or replacing these layers with an SVM. Overall, 11 configurations based on these methods have been tested: 8 transfer learning-based, and 3 TactNet-based. Moreover, a study of the performance of the methods and a comparative discussion with the current state-of-the-art on tactile object recognition is presented.

Mattia Leonori, Juan M. Gandarias, Arash Ajoudani

Safety is one of the most fundamental aspects of robotics, especially when it comes to collaborative robots (cobots) that are expected to physically interact with humans. Although a large body of literature has focused on safety-related aspects for fixed-based cobots, a low effort has been put into developing collaborative mobile manipulators. In response to this need, this work presents MOCA-S, i.e., Sensitive Mobile Collaborative Robotic Assistant, that integrates a low-cost, capacitive tactile cover to measure interaction forces applied to the robot base. The tactile cover comprises a set of 11 capacitive large-area tactile sensors distributed as a 1-D tactile array around the base. Characterization of the tactile sensors with different materials is included. Moreover, two expanded whole-body controllers that exploit the platform's tactile cover and the loco-manipulation features are proposed. These controllers are tested in two experiments, demonstrating the potential of MOCA-S for safe physical Human-Robot Interaction (pHRI). Finally, an experiment is carried out in which an undesired collision occurs between MOCA-S and a human during a loco-manipulation task. The results demonstrate the intrinsic safety of MOCA-S and the proposed controllers, suggesting a new step towards creating safe mobile manipulators.

Alberto Giammarino, Juan M. Gandarias, Pietro Balatti et al.

This paper presents a mobile supernumerary robotic approach to physical assistance in human-robot conjoined actions. The study starts with a description of the SUPER-MAN concept. The idea is to develop and utilize mobile collaborative systems that can follow human loco-manipulation commands to perform industrial tasks through three main components: i) an admittance-type interface, ii) a human-robot interaction controller, and iii) a supernumerary robotic body. Next, we present two possible implementations within the framework from theoretical and hardware perspectives. The first system is called MOCA-MAN and comprises a redundant torque-controlled robotic arm and an omnidirectional mobile platform. The second one is called Kairos-MAN, formed by a high-payload 6-DoF velocity-controlled robotic arm and an omnidirectional mobile platform. The systems share the same admittance interface, through which user wrenches are translated to loco-manipulation commands generated by whole-body controllers of each system. Besides, a thorough user study with multiple and cross-gender subjects is presented to reveal the quantitative performance of the two systems in effort-demanding and dexterous tasks. Moreover, we provide qualitative results from the NASA-TLX questionnaire to demonstrate the SUPER-MAN approach's potential and its acceptability from the users' viewpoint.

Francisco J. Ruiz-Ruiz, Alberto Giammarino, Marta Lorenzini et al.

This paper presents the design and development of a robotic system to give physical assistance to the elderly or people with neurological disorders such as Ataxia or Parkinson's. In particular, we propose using a mobile collaborative robot with an interaction-assistive whole-body interface to help people unable to maintain balance. The robotic system consists of an Omni-directional mobile base, a high-payload robotic arm, and an admittance-type interface acting as a support handle while measuring human-sourced interaction forces. The postural balance of the human body is estimated through the projection of the body Center of Mass (CoM) to the support polygon (SP) representing the quasi-static Center of Pressure (CoP). In response to the interaction forces and the tracking of the human posture, the robot can create assistive forces to restore balance in case of its loss. Otherwise, during normal stance or walking, it will follow the user with minimum/no opposing forces through the generation of coupled arm and base movements. As the balance-restoring strategy, we propose two strategies and evaluate them in a laboratory setting on healthy human participants. Quantitative and qualitative results of a 12-subjects experiment are then illustrated and discussed, comparing the performances of the two strategies and the overall system.

Juan M. Gandarias, Pietro Balatti, Edoardo Lamon et al.

This paper presents a physical interface for collaborative mobile manipulators in industrial manufacturing and logistics applications. The proposed work builds on our earlier MOCA-MAN interface, through which an operator could be physically coupled to a mobile manipulator to be assisted in performing daily activities. The previous interface was based on a magnetic clamp attached to one arm of the user for the coupling stage, and a bracelet based on EMG sensors on the other arm for human-robot communication via gestures. The new interface instead presents the following additions: i) An industrial-like design that allows the worker to couple/decouple easily and to operate mobile manipulators locally; ii) A simplistic communication channel via a simple buttons board that allows controlling the robot with one hand only; iii) The interface offers enhanced loco-manipulation capabilities that do not compromise the worker mobility. In addition, an experimental evaluation with six human subjects is carried out to analyze the enhanced locomotion and flexibility of the proposed interface in terms of mobility constraint, usability, and physical load reduction.