Ye Zhao

Yunhai Han, Kelin Yu, Rahul Batra et al.

Reliable robotic grasping, especially with deformable objects such as fruits, remains a challenging task due to underactuated contact interactions with a gripper, unknown object dynamics and geometries. In this study, we propose a Transformer-based robotic grasping framework for rigid grippers that leverage tactile and visual information for safe object grasping. Specifically, the Transformer models learn physical feature embeddings with sensor feedback through performing two pre-defined explorative actions (pinching and sliding) and predict a grasping outcome through a multilayer perceptron (MLP) with a given grasping strength. Using these predictions, the gripper predicts a safe grasping strength via inference. Compared with convolutional recurrent networks, the Transformer models can capture the long-term dependencies across the image sequences and process spatial-temporal features simultaneously. We first benchmark the Transformer models on a public dataset for slip detection. Following that, we show that the Transformer models outperform a CNN+LSTM model in terms of grasping accuracy and computational efficiency. We also collect a new fruit grasping dataset and conduct online grasping experiments using the proposed framework for both seen and unseen fruits. {In addition, we extend our model to objects with different shapes and demonstrate the effectiveness of our pre-trained model trained on our large-scale fruit dataset. Our codes and dataset are public on GitHub.

Ye Zhao

Humanoid robots are increasingly demanded to operate in interactive and human-surrounded environments while achieving sophisticated locomotion and manipulation tasks. To accomplish these tasks, roboticists unremittingly seek for advanced methods that generate whole-body coordination behaviors and meanwhile fulfill various planning and control objectives. Undoubtedly, these goals pose fundamental challenges to the robotics and control community. To take an incremental step towards reducing the performance gap between theoretical foundations and real implementations, we present a planning and control framework for the humanoid, especially legged robots, for achieving high performance and generating agile motions. A particular concentration is on the robust, optimal and real-time performance. This framework constitutes three hierarchical layers: First, we present a robust optimal phase-space planning framework for dynamic legged locomotion over rough terrain. This framework is a hybrid motion planner incorporating a series of pivotal components. Second, we take a step toward formally synthesizing high-level reactive planners for whole-body locomotion in constrained environments. We formulate a two-player temporal logic game between the contact planner and its possibly-adversarial environment. Third, we propose a distributed control architecture for the latency-prone humanoid robotic systems. A central experimental phenomenon is observed that the stability of high impedance distributed controllers is highly sensitive to damping feedback delay but much less to stiffness feedback delay. We pursue a detailed analysis of the distributed controllers where damping feedback effort is executed in proximity to the control plant, and stiffness feedback effort is implemented in a latency-prone centralized control process.