Mark Yim

Victoria A. Webster-Wood, Nicholas Gravish, Amir Alavi et al.

This report provides a summary of the outcomes of the Interdisciplinary Workshop on Mechanical Intelligence held in 2024. Mechanical Intelligence (MI) represents the phenomenon that novel structural features of material/biological/robotic systems can encode intelligence through responsiveness, adaptivity, memory, and learning in the mechanical structure itself. This is in contrast to computational intelligence, wherein the intelligence functions occur through electrical signaling and computer code. The two-day workshop was held at NSF headquarters on May 30-31 and included 38 invited academic researcher participants, and 8 program officers from the NSF. The workshop was structured around active small and large group discussions in groups of 4-5 and 9-10 with the goal of addressing topical questions on MI. Working groups entered notes into shared presentation slides for each discussion session and presented their outcomes in a final presentation on the last day. Here we summarize the overall outcomes of the workshop.

Jessica Yin, Haozhi Qi, Jitendra Malik et al.

Recent progress in reinforcement learning (RL) and tactile sensing has significantly advanced dexterous manipulation. However, these methods often utilize simplified tactile signals due to the gap between tactile simulation and the real world. We introduce a sensor model for tactile skin that enables zero-shot sim-to-real transfer of ternary shear and binary normal forces. Using this model, we develop an RL policy that leverages sliding contact for dexterous in-hand translation. We conduct extensive real-world experiments to assess how tactile sensing facilitates policy adaptation to various unseen object properties and robot hand orientations. We demonstrate that our 3-axis tactile policies consistently outperform baselines that use only shear forces, only normal forces, or only proprioception. Website: https://jessicayin.github.io/tactile-skin-rl/

Andrew Jang-Ho Bae, Myeongjin Choi, Haorui Li et al.

This paper presents an object manipulation strategy for the Variable Topology Truss (VTT) system, a truss robot that comprises actuated truss members connected by passive spherical joints. Although truss robots were originally proposed as rapidly deployable manipulators, manipulation strategy has not been studied thoroughly. To enable manipulation, we introduce a hybrid control framework that regulates position and force concurrently without explicit decoupling. At the actuator level, each member employs a sensor-based force feedback controller to generate the desired axial forces despite high actuator friction. At the task level, the forces applied at the end-effector nodes are produced by computing the required member forces using a static model of the VTT. We evaluate force-tracking performance through experiments on both a single member module and the full VTT system. Finally, we demonstrate object manipulation using two representative configurations and quantitatively assess combined position and force tracking performance. Experimental results confirm that the proposed approach enables consistent and reliable object manipulation with the VTT system.

Gregory M. Campbell, Gentian Muhaxheri, Leonardo Ferreira Guilhoto et al.

Soft pneumatic actuators (SPA) made from elastomeric materials can provide large strain and large force. The behavior of locally strain-restricted hyperelastic materials under inflation has been investigated thoroughly for shape reconfiguration, but requires further investigation for trajectories involving external force. In this work we model force-pressure-height relationships for a concentrically strain-limited class of soft pneumatic actuators and demonstrate the use of this model to design SPA response for object lifting. We predict relationships under different loadings by solving energy minimization equations and verify this theory by using an automated test rig to collect rich data for n=22 Ecoflex 00-30 membranes. We collect this data using an active learning pipeline to efficiently model the design space. We show that this learned material model outperforms the theory-based model and naive curve-fitting approaches. We use our model to optimize membrane design for different lift tasks and compare this performance to other designs. These contributions represent a step towards understanding the natural response for this class of actuator and embodying intelligent lifts in a single-pressure input actuator system.

Chao Liu, Tarik Tosun, Mark Yim

Accurate position sensing is important for state estimation and control in robotics. Reliable and accurate position sensors are usually expensive and difficult to customize. Incorporating them into systems that have very tight volume constraints such as modular robots are particularly difficult. PaintPots are low-cost, reliable, and highly customizable position sensors, but their performance is highly dependent on the manufacturing and calibration process. This paper presents a Kalman filter with a simplified observation model developed to deal with the non-linearity issues that result in the use of low-cost microcontrollers. In addition, a complete solution for the use of PaintPots in a variety of sensing modalities including manufacturing, characterization, and estimation is presented for an example modular robot, SMORES-EP. This solution can be easily adapted to a wide range of applications.

Walker Gosrich, Siddharth Mayya, Rebecca Li et al.

This paper develops a decentralized approach to mobile sensor coverage by a multi-robot system. We consider a scenario where a team of robots with limited sensing range must position itself to effectively detect events of interest in a region characterized by areas of varying importance. Towards this end, we develop a decentralized control policy for the robots -- realized via a Graph Neural Network -- which uses inter-robot communication to leverage non-local information for control decisions. By explicitly sharing information between multi-hop neighbors, the decentralized controller achieves a higher quality of coverage when compared to classical approaches that do not communicate and leverage only local information available to each robot. Simulated experiments demonstrate the efficacy of multi-hop communication for multi-robot coverage and evaluate the scalability and transferability of the learning-based controllers.

Andrew Specian, Ross Mead, Simon Kim et al.

Hardware platforms for socially interactive robotics can be limited by cost or lack of functionality. This paper presents the overall system -- design, hardware, and software -- for Quori, a novel, affordable, socially interactive humanoid robot platform for facilitating non-contact human-robot interaction (HRI) research. The design of the system is motivated by feedback sampled from the HRI research community. The overall design maintains a balance of affordability and functionality. Initial Quori testing and a six-month deployment are presented. Ten Quori platforms have been awarded to a diverse group of researchers from across the United States to facilitate HRI research to build a community database from a common platform.

Chao Liu, Sencheng Yu, Mark Yim

Truss robots are highly redundant parallel robotic systems that can be applied in a variety of scenarios. The variable topology truss (VTT) is a class of modular truss robots. As self-reconfigurable modular robots, a VTT is composed of many edge modules that can be rearranged into various structures depending on the task. These robots change their shape by not only controlling joint positions as with fixed morphology robots, but also reconfiguring the connectivity between truss members in order to change their topology. The motion planning problem for VTT robots is difficult due to their varying morphology, high dimensionality, the high likelihood for self-collision, and complex motion constraints. In this paper, a new motion planning framework to dramatically alter the structure of a VTT is presented. It can also be used to solve locomotion tasks that are much more efficient compared with previous work. Several test scenarios are used to show its effectiveness. Supplementary materials are available at https://www.modlabupenn.org/vtt-motion-planning/.

Gedaliah Knizhnik, Mark Yim

The Modboat is an autonomous surface robot that turns the oscillation of a single motor into a controlled paddling motion through passive flippers. Inertial control methods developed in prior work can successfully drive the Modboat along trajectories and enable docking to neighboring modules, but have a non-constant cycle time and cannot react to dynamic environments. In this work we present a thrust direction control method for the Modboat that significantly improves the time-response of the system and increases the accuracy with which it can be controlled. We experimentally demonstrate that this method can be used to perform more compact maneuvers than prior methods or comparable robots can. We also present an extension to the controller that solves the reaction wheel problem of unbounded actuator velocity, and show that it further improves performance.

Chao Liu, Mark Yim

Motion planning in high-dimensional space is a challenging task. In order to perform dexterous manipulation in an unstructured environment, a robot with many degrees of freedom is usually necessary, which also complicates its motion planning problem. Real-time control brings about more difficulties in which robots have to maintain the stability while moving towards the target. Redundant systems are common in modular robots that consist of multiple modules and are able to transformed into different configurations with respect to different needs. Different from robots with fixed geometry or configurations, the kinematics model of a modular robotic system can alter as the robot reconfigures itself, and developing a generic control and motion planning approach for such systems is difficult, especially when multiple motion goals are coupled. A new manipulation planning framework is developed in this paper. The problem is formulated as a sequential linearly constrained quadratic program (QP) that can be solved efficiently. Some constraints can be incorporated into this QP, including a novel way to approximate environment obstacles. This solution can be used directly for real-time applications or as an off-line planning tool, and it is validated and demonstrated on the CKBot and SMORES-EP modular robot platforms.

Chao Liu, Qian Lin, Hyun Kim et al.

Self-assembly of modular robotic systems enables the construction of complex robotic configurations to adapt to different tasks. This paper presents a framework for SMORES types of modular robots to efficiently self-assemble into tree topologies. These modular robots form kinematic chains that have been shown to be capable of a large variety of manipulation and locomotion tasks, yet they can reconfigure using a mobile reconfiguration. A desired kinematic topology can be mapped onto a planar pattern with optimal module assignment based on the modules' locations, then the mobile reconfiguration assembly process can be executed in parallel. A docking controller is developed to guarantee the success of docking processes. A hybrid control architecture is designed to handle a large number of modules and complex behaviors of each individual, and achieve efficient and robust self-assembly actions. The framework is demonstrated in both hardware and simulation on the SMORES-EP platform.

Gedaliah Knizhnik, Mark Yim

We describe a docking mechanism and strategy to allow modular self-assembly for the Modboat: an inexpensive underactuated oscillating swimming robot powered by a single motor. Because propulsion is achieved through oscillation, orientation can be controlled only in the average; this complicates docking, which requires precise position and orientation control. Given these challenges, we present a docking strategy and a motion primitive for controlling orientation, and show that this strategy allows successful docking in multiple configurations. Moreover, we demonstrate that the Modboat is also capable of undocking and changing its dock configuration, all without any additional actuation. This is unique among similar modular robotic systems.

Gedaliah Knizhnik, Mark Yim

We present a novel design for a low-cost robotic boat powered by a single actuator, useful for both modular and swarming applications. The boat uses the conservation of angular momentum and passive flippers to convert the motion of a single motor into an adjustable paddling motion for propulsion and steering. We develop design criteria for modularity and swarming and present a prototype implementing these criteria. We identify significant mechanical sensitivities with the presented design, theorize about the cause of the sensitivities, and present an improved design for future work.

Tarik Tosun, Cynthia Sung, Colin McCloskey et al.

Building structures can allow a robot to surmount large obstacles, expanding the set of areas it can reach. This paper presents a planning algorithm to automatically determine what structures a construction-capable robot must build in order to traverse its entire environment. Given an environment, a set of building blocks, and a robot capable of building structures, we seek a optimal set of structures (using a minimum number of building blocks) that could be built to make the entire environment traversable with respect to the robot's movement capabilities. We show that this problem is NP-Hard, and present a complete, optimal algorithm that solves it using a branch-and-bound strategy. The algorithm runs in exponential time in the worst case, but solves typical problems with practical speed. In hardware experiments, we show that the algorithm solves 3D maps of real indoor environments in about one minute, and that the structures selected by the algorithm allow a robot to traverse the entire environment. An accompanying video is available online at https://youtu.be/B9WM557NP44.

Gangyuan Jing, Tarik Tosun, Mark Yim et al.

The advantage of modular self-reconfigurable robot systems is their flexibility, but this advantage can only be realized if appropriate configurations (shapes) and behaviors (controlling programs) can be selected for a given task. In this paper, we present an integrated system for addressing high-level tasks with modular robots, and demonstrate that it is capable of accomplishing challenging, multi-part tasks in hardware experiments. The system consists of four tightly integrated components: (1) A high-level mission planner, (2) A large design library spanning a wide set of functionality, (3) A design and simulation tool for populating the library with new configurations and behaviors, and (4) modular robot hardware. This paper builds on earlier work by the authors, extending the original system to include environmentally adaptive parametric behaviors, which integrate motion planners and feedback controllers with the system.

Tarik Tosun, Jonathan Daudelin, Gangyuan Jing et al.

We present a system enabling a modular robot to autonomously build structures in order to accomplish high-level tasks. Building structures allows the robot to surmount large obstacles, expanding the set of tasks it can perform. This addresses a common weakness of modular robot systems, which often struggle to traverse large obstacles. This paper presents the hardware, perception, and planning tools that comprise our system. An environment characterization algorithm identifies features in the environment that can be augmented to create a path between two disconnected regions of the environment. Specially-designed building blocks enable the robot to create structures that can augment the environment to make obstacles traversable. A high-level planner reasons about the task, robot locomotion capabilities, and environment to decide if and where to augment the environment in order to perform the desired task. We validate our system in hardware experiments

Jonathan Daudelin, Gangyuan Jing, Tarik Tosun et al.

The theoretical ability of modular robots to reconfigure in response to complex tasks in a priori unknown environments has frequently been cited as an advantage and remains a major motivator for work in the field. We present a modular robot system capable of autonomously completing high-level tasks by reactively reconfiguring to meet the needs of a perceived, a priori unknown environment. The system integrates perception, high-level planning, and modular hardware, and is validated in three hardware demonstrations. Given a high-level task specification, a modular robot autonomously explores an unknown environment, decides when and how to reconfigure, and manipulates objects to complete its task. The system architecture balances distributed mechanical elements with centralized perception, planning, and control. By providing an example of how a modular robot system can be designed to leverage reactive reconfigurability in unknown environments, we have begun to lay the groundwork for modular self-reconfigurable robots to address tasks in the real world.