Nabil Simaan

Samrat Bhattacharyya, Nabil Simaan

This paper presents an online path planning algorithm for safe autonomous manipulation of a flexibly constrained object in an unknown environment. Methods for real time identification and characterization of perceived flexible constraints and global stiffness are presented. Used in tandem, these methods allow a robot to simultaneously explore, characterize, and manipulate an elastic system safely. Navigation without a-priori knowledge of the system is achieved using constraint exploration based on local force and position information. The perceived constraint stiffness is considered at multiple poses along an object's (system) trajectory. Using stiffness eigenvector information, global stiffness behavior is characterized and identified using an atlas of simple mechanical constraints, such as hinges and planar constraints. Validation of these algorithms is carried out by simulation and experimentally. The ability to recognize several common simple mechanical constraints (such as a flexible hinge) in real time, and to subsequently identify relevant screw parameters is demonstrated. These results suggest the feasibility of simultaneous global constrain/stiffness exploration and safe manipulation of flexibly constrained objects. We believe that this approach will eventually enable safe cooperative manipulation in applications such as organ retraction and manipulation during surgery

Ashkan Shahbazi, Elaheh Akbari, Kyvia Pereira et al.

We introduce SurgFormer, a multiresolution gated transformer for data driven soft tissue simulation on volumetric meshes. High fidelity biomechanical solvers are often too costly for interactive use, so we train SurgFormer on solver generated data to predict nodewise displacement fields at near real time rates. SurgFormer builds a fixed mesh hierarchy and applies repeated multibranch blocks that combine local message passing, coarse global self attention, and pointwise feedforward updates, fused by learned per node, per channel gates to adaptively integrate local and long range information while remaining scalable on large meshes. For cut conditioned simulation, resection information is encoded as a learned cut embedding and provided as an additional input, enabling a unified model for both standard deformation prediction and topology altering cases. We also introduce two surgical simulation datasets generated under a unified protocol with XFEM based supervision: a cholecystectomy resection dataset and an appendectomy manipulation and resection dataset with cut and uncut cases. To our knowledge, this is the first learned volumetric surrogate setting to study XFEM supervised cut conditioned deformation within the same volumetric pipeline as standard deformation prediction. Across diverse baselines, SurgFormer achieves strong accuracy with favorable efficiency, making it a practical backbone for both tasks. {Code, data, and project page: \href{https://mint-vu.github.io/SurgFormer/}{available here}}

Jared Lawson, Rohan Chitale, Nabil Simaan

Safe navigation of steerable and robotic catheters in the cerebral vasculature requires awareness of the catheters shape and pose. Currently, a significant perception burden is placed on interventionalists to mentally reconstruct and predict catheter motions from biplane fluoroscopy images. Efforts to track these catheters are limited to planar segmentation or bulky sensing instrumentation, which are incompatible with microcatheters used in neurointervention. In this work, a catheter is equipped with custom radiopaque markers arranged to enable simultaneous shape and pose estimation under biplane fluoroscopy. A design measure is proposed to guide the arrangement of these markers to minimize sensitivity to marker tracking uncertainty. This approach was deployed for microcatheters smaller than 2mm OD navigating phantom vasculature with shape tracking errors less than 1mm and catheter roll errors below 40 degrees. This work can enable steerable catheters to autonomously navigate under biplane imaging.

Jumanh Atoum, Garrison L. H. Johnston, Nabil Simaan et al.

Recognizing surgical gestures in real-time is a stepping stone towards automated activity recognition, skill assessment, intra-operative assistance, and eventually surgical automation. The current robotic surgical systems provide us with rich multi-modal data such as video and kinematics. While some recent works in multi-modal neural networks learn the relationships between vision and kinematics data, current approaches treat kinematics information as independent signals, with no underlying relation between tool-tip poses. However, instrument poses are geometrically related, and the underlying geometry can aid neural networks in learning gesture representation. Therefore, we propose combining motion invariant measures (curvature and torsion) with vision and kinematics data using a relational graph network to capture the underlying relations between different data streams. We show that gesture recognition improves when combining invariant signals with tool position, achieving 90.3\% frame-wise accuracy on the JIGSAWS suturing dataset. Our results show that motion invariant signals coupled with position are better representations of gesture motion compared to traditional position and quaternion representations. Our results highlight the need for geometric-aware modeling of kinematics for gesture recognition.

Neel Shihora, Rashid M. Yasin, Ryan Walsh et al.

Cricothyrotomy is a life-saving emergency intervention that secures an alternate airway route after a neck injury or obstruction. The procedure starts with identifying the correct location (the cricothyroid membrane) for creating an incision to insert an endotracheal tube. This location is determined using a combination of visual and palpation cues. Enabling robot-assisted remote cricothyrotomy may extend this life-saving procedure to injured soldiers or patients who may not be readily accessible for on-site intervention during search-and-rescue scenarios. As a first step towards achieving this goal, this paper explores the feasibility of palpation-assisted landmark identification for cricothyrotomy. Using a cricothyrotomy training simulator, we explore several alternatives for in-situ remote localization of the cricothyroid membrane. These alternatives include a) unaided telemanipulation, b) telemanipulation with direct force feedback, c) telemanipulation with superimposed motion excitation for online stiffness estimation and display, and d) fully autonomous palpation scan initialized based on the user's understanding of key anatomical landmarks. Using the manually digitized cricothyroid membrane location as ground truth, we compare these four methods for accuracy and repeatability of identifying the landmark for cricothyrotomy, time of completion, and ease of use. These preliminary results suggest that the accuracy of remote cricothyrotomy landmark identification is improved when the user is aided with visual and force cues. They also show that, with proper user initialization, landmark identification using remote palpation is feasible - therefore satisfying a key pre-requisite for future robotic solutions for remote cricothyrotomy.

Garrison L. H. Johnston, Andrew L. Orekhov, Nabil Simaan

Collaborative robots should ideally use low torque actuators for passive safety reasons. However, some applications require these collaborative robots to reach deep into confined spaces while assisting a human operator in physically demanding tasks. In this paper, we consider the use of in-situ collaborative robots (ISCRs) that balance the conflicting demands of passive safety dictating low torque actuation and the need to reach into deep confined spaces. We consider the judicious use of bracing as a possible solution to these conflicting demands and present a modeling framework that takes into account the constrained kinematics and the effect of bracing on the end-effector compliance. We then define a redundancy resolution framework that minimizes the directional compliance of the end-effector while maximizing end-effector dexterity. Kinematic simulation results show that the redundancy resolution strategy successfully decreases compliance and improves kinematic conditioning while satisfying the constraints imposed by the bracing task. Applications of this modeling framework can support future research on the choice of bracing locations and support the formation of an admittance control framework for collaborative control of ISCRs under bracing constraints. Such robots can benefit workers in the future by reducing the physiological burdens that contribute to musculoskeletal injury.

Andrew L. Orekhov, Nabil Simaan

Choosing a kinematic model for a continuum robot typically involves making a tradeoff between accuracy and computational complexity. One common modeling approach is to use the Cosserat rod equations, which have been shown to be accurate for many types of continuum robots. This approach, however, still presents significant computational cost, particularly when many Cosserat rods are coupled via kinematic constraints. In this work, we propose a numerical method that combines orthogonal collocation on the local rod curvature and forward integration of the Cosserat rod kinematic equations via the Magnus expansion, allowing the equilibrium shape to be written as a product of matrix exponentials. We provide a bound on the maximum step size to guarantee convergence of the Magnus expansion for the case of Cosserat rods, compare in simulation against other approaches, and demonstrate the tradeoffs between speed and accuracy for the fourth and sixth order Magnus expansions as well as for different numbers of collocation points. Our results show that the proposed method can find accurate solutions to the Cosserat rod equations and can potentially be competitive in computation speed.

Yue Chen, Long Wang, Kevin Galloway et al.

Soft robots offer an alternative approach to manipulate inside the constrained space while maintaining the safe interaction with the external environment. Due to its adaptable compliance characteristic, external contact force can easily deform the robot shapes and lead to undesired robot kinematic and dynamic properties. Accurate contact detection and contact position estimation are of critical importance for soft robot modeling, control, trajectory planning, and eventually affect the success of task completion. In this paper, we focus on the study of 1-DoF soft pneumatic bellow bending actuator, which is one of the fundamental components to construct complex, multi-DoF soft robots. This 1-DoF soft robot is modeled through the integral representation of the spacial curve. The direct and instantaneous kinematics are calculated explicitly through a modal method. The fixed centrode deviation (FCD) method is used to to detect the external contact and estimate contact location. Simulation results indicate that the contact location can be accurately estimated by solving a nonlinear least square optimization problem.

Andrew L. Orekhov, Colette Abah, Nabil Simaan

The surgical paradigm of Minimally Invasive Surgery (MIS) has been a key driver to the adoption of robotic surgical assistance. Progress in the last three decades has led to a gradual transition from manual laparoscopic surgery with rigid instruments to robot-assisted surgery. In the last decade, the increasing demand for new surgical paradigms to enable access into the anatomy without skin incision (intraluminal surgery) or with a single skin incision (Single Port Access surgery - SPA) has led researchers to investigate snake-like flexible surgical devices. In this chapter, we first present an overview of the background, motivation, and taxonomy of MIS and its newer derivatives. Challenges of MIS and its newer derivatives (SPA and intraluminal surgery) are outlined along with the architectures of new snake-like robots meeting these challenges. We also examine the commercial and research surgical platforms developed over the years, to address the specific functional requirements and constraints imposed by operations in confined spaces. The chapter concludes with an evaluation of open problems in surgical robotics for intraluminal and SPA, and a look at future trends in surgical robot design that could potentially address these unmet needs.

Long Wang, Giuseppe Del Giudice, Nabil Simaan

Recently, a new concept for continuum robots capable of producing macro-scale and micro-scale motion has been presented. These robots achieve their multi-scale motion capabilities by coupling direct-actuation of push-pull back-bones for macro motion with indirect actuation whereby the equilibrium pose is altered to achieve micro-scale motion. This paper presents a first attempt at explaining the micro-motion capabilities of these robots from a modeling perspective. This paper presents the macro and micro motion kinematics of a single segment continuum robot by using statics coupling effects among its sub-segments. Experimental observations of the micro-scale motion demonstrate a turning point behavior which could not be explained well using the current modeling methods. We present a simplistic modeling approach that introduces two calibration parameters to calibrate the moment coupling effects among the sub segments of the robot. It is shown that these two parameters can reproduce the turning point behavior at the micro-scale. The instantaneous macro and micro scale kinematics Jacobians and the calibration parameters identification Jacobian are derived. The modeling approach is verified against experimental data showing that our simplistic modeling approach can capture the experimental motion data with RMS position error of 5.82 micrometers if one wishes to fit the entire motion profile with the turning point. If one chooses to exclude motions past the turning point, our model can fit the experimental data with an accuracy of 4.76 micrometers.

Nabil Simaan, Rashid M. Yasin, Long Wang

Emerging paradigms furthering the reach of medical technology deeper into human anatomy present unique modeling, control and sensing problems. This paper discusses a brief history of medical robotics leading to the current trend of minimally invasive intervention and diagnostics in confined spaces. Robotics for natural orifice and single port access surgery, capsule and magnetically actuated robotics and microrobotics are discussed with the aim of elucidating the state of the art. Works on modeling, sensing and control of mechanical architectures of robots for natural orifice and single port access surgery are discussed, followed by a presentation of works on magnetic actuation, sensing and localization for capsule robotics and microrobotics. Finally challenges and open problems in each one of these areas are presented.

Elif Ayvali, Rangaprasad Arun Srivatsan, Long Wang et al.

One of the goals of computer-aided surgery is to match intraoperative data to preoperative images of the anatomy and add complementary information that can facilitate the task of surgical navigation. In this context, mechanical palpation can reveal critical anatomical features such as arteries and cancerous lumps which are stiffer that the surrounding tissue. This work uses position and force measurements obtained during mechanical palpation for registration and stiffness mapping. Prior approaches, including our own, exhaustively palpated the entire organ to achieve this goal. To overcome the costly palpation of the entire organ, a Bayesian optimization framework is introduced to guide the end effector to palpate stiff regions while simultaneously updating the registration of the end effector to an a priori geometric model of the organ, hence enabling the fusion of ntraoperative data into the a priori model obtained through imaging. This new framework uses Gaussian processes to model the stiffness distribution and Bayesian optimization to direct where to sample next for maximum information gain. The proposed method was evaluated with experimental data obtained using a Cartesian robot interacting with a silicone organ model and an ex vivo porcine liver.