A data-set of piercing needle through deformable objects for Deep Learning from Demonstrations
This dataset addresses the challenge of automating robotic suturing with minimally invasive robots, which is a significant problem for medical robotics and surgical automation, by providing a resource for training Deep Neural Networks.
This paper introduces a dataset for learning robotic needle piercing through deformable objects using Deep Learning from Demonstrations (Deep-RLfD). The dataset comprises 60 successful needle insertion trials with a da Vinci Research Kit, including synchronized high-resolution camera footage, robot data, and control inputs. The authors also implemented baseline Deep-RLfD architectures, finding that Recurrent Convolutional Networks (RCNs) improved prediction accuracy compared to feed-forward CNNs.
Many robotic tasks are still teleoperated since automating them is very time consuming and expensive. Robot Learning from Demonstrations (RLfD) can reduce programming time and cost. However, conventional RLfD approaches are not directly applicable to many robotic tasks, e.g. robotic suturing with minimally invasive robots, as they require a time-consuming process of designing features from visual information. Deep Neural Networks (DNN) have emerged as useful tools for creating complex models capturing the relationship between high-dimensional observation space and low-level action/state space. Nonetheless, such approaches require a dataset suitable for training appropriate DNN models. This paper presents a dataset of inserting/piercing a needle with two arms of da Vinci Research Kit in/through soft tissues. The dataset consists of (1) 60 successful needle insertion trials with randomised desired exit points recorded by 6 high-resolution calibrated cameras, (2) the corresponding robot data, calibration parameters and (3) the commanded robot control input where all the collected data are synchronised. The dataset is designed for Deep-RLfD approaches. We also implemented several deep RLfD architectures, including simple feed-forward CNNs and different Recurrent Convolutional Networks (RCNs). Our study indicates RCNs improve the prediction accuracy of the model despite that the baseline feed-forward CNNs successfully learns the relationship between the visual information and the next step control actions of the robot. The dataset, as well as our baseline implementations of RLfD, are publicly available for bench-marking at https://github.com/imanlab/d-lfd.