Alfonso García-Cerezo

Francisco Pastor, Jorge García-González, Juan M. Gandarias et al.

Recent advances in the field of intelligent robotic manipulation pursue providing robotic hands with touch sensitivity. Haptic perception encompasses the sensing modalities encountered in the sense of touch (e.g., tactile and kinesthetic sensations). This letter focuses on multimodal object recognition and proposes analytical and data-driven methodologies to fuse tactile- and kinesthetic-based classification results. The procedure is as follows: a three-finger actuated gripper with an integrated high-resolution tactile sensor performs squeeze-and-release Exploratory Procedures (EPs). The tactile images and kinesthetic information acquired using angular sensors on the finger joints constitute the time-series datasets of interest. Each temporal dataset is fed to a Long Short-term Memory (LSTM) Neural Network, which is trained to classify in-hand objects. The LSTMs provide an estimation of the posterior probability of each object given the corresponding measurements, which after fusion allows to estimate the object through Bayesian and Neural inference approaches. An experiment with 36-classes is carried out to evaluate and compare the performance of the fused, tactile, and kinesthetic perception systems.The results show that the Bayesian-based classifiers improves capabilities for object recognition and outperforms the Neural-based approach.

Raul Castilla-Arquillo, Carlos Perez-del-Pulgar, Levin Gerdes et al.

Robot navigation in unstructured environments requires multimodal perception systems that can support safe navigation. Multimodality enables the integration of complementary information collected by different sensors. However, this information must be processed by machine learning algorithms specifically designed to leverage heterogeneous data. Furthermore, it is necessary to identify which sensor modalities are most informative for navigation in the target environment. In Martian exploration, thermal imagery has proven valuable for assessing terrain safety due to differences in thermal behaviour between soil types. This work presents OmniUnet, a transformer-based neural network architecture for semantic segmentation using RGB, depth, and thermal (RGB-D-T) imagery. A custom multimodal sensor housing was developed using 3D printing and mounted on the Martian Rover Testbed for Autonomy (MaRTA) to collect a multimodal dataset in the Bardenas semi-desert in northern Spain. This location serves as a representative environment of the Martian surface, featuring terrain types such as sand, bedrock, and compact soil. A subset of this dataset was manually labeled to support supervised training of the network. The model was evaluated both quantitatively and qualitatively, achieving a pixel accuracy of 80.37% and demonstrating strong performance in segmenting complex unstructured terrain. Inference tests yielded an average prediction time of 673 ms on a resource-constrained computer (Jetson Orin Nano), confirming its suitability for on-robot deployment. The software implementation of the network and the labeled dataset have been made publicly available to support future research in multimodal terrain perception for planetary robotics.

Juan M. Gandarias, Alfonso J. García-Cerezo, Jesús M. Gómez-de-Gabriel

Novel high-resolution pressure-sensor arrays allow treating pressure readings as standard images. Computer vision algorithms and methods such as Convolutional Neural Networks (CNN) can be used to identify contact objects. In this paper, a high-resolution tactile sensor has been attached to a robotic end-effector to identify contacted objects. Two CNN-based approaches have been employed to classify pressure images. These methods include a transfer learning approach using a pre-trained CNN on an RGB-images dataset and a custom-made CNN (TactNet) trained from scratch with tactile information. The transfer learning approach can be carried out by retraining the classification layers of the network or replacing these layers with an SVM. Overall, 11 configurations based on these methods have been tested: 8 transfer learning-based, and 3 TactNet-based. Moreover, a study of the performance of the methods and a comparative discussion with the current state-of-the-art on tactile object recognition is presented.

J. Ricardo Sánchez-Ibáñez, Carlos J. Pérez-del-Pulgar, Javier Serón et al.

The optimal traverse of irregular terrains made by ground mobile robots heavily depends on the adequacy of the cost models used to plan the path they follow. The criteria to define optimality may be based on minimizing energy consumption and/or preserving the robot stability. This entails the proper assessment of anisotropy to account for the robot driving on top of slopes with different directions. To fulfill this demand, this paper presents the Continuous Anisotropic Model for Inclined Surfaces, a cost model compatible with anisotropic path planners like the bi-directional Ordered Upwind Method. This model acknowledges how the orientation of the robot with respect to any slope determines its energetic cost, considering the action of gravity and terramechanic effects such as the slippage. Moreover, the proposed model can be tuned to define a trade-off between energy minimization and Roll angle reduction. The results from two simulation tests demonstrate how, to find the optimal path in scenarios containing slopes, in certain situations the use of this model can be more advantageous than relying on isotropic cost functions. Finally, the outcome of a field experiment involving a skid-steering robot that drives on top of a real slope is also discussed.