G. C. Nandi

Priya Shukla, Vandana Kushwaha, G. C. Nandi

Grasping objects intelligently is a challenging task even for humans and we spend a considerable amount of time during our childhood to learn how to grasp objects correctly. In the case of robots, we can not afford to spend that much time on making it to learn how to grasp objects effectively. Therefore, in the present research we propose an efficient learning architecture based on VQVAE so that robots can be taught with sufficient data corresponding to correct grasping. However, getting sufficient labelled data is extremely difficult in the robot grasping domain. To help solve this problem, a semi-supervised learning based model which has much more generalization capability even with limited labelled data set, has been investigated. Its performance shows 6\% improvement when compared with existing state-of-the-art models including our earlier model. During experimentation, It has been observed that our proposed model, RGGCNN2, performs significantly better, both in grasping isolated objects as well as objects in a cluttered environment, compared to the existing approaches which do not use unlabelled data for generating grasping rectangles. To the best of our knowledge, developing an intelligent robot grasping model (based on semi-supervised learning) trained through representation learning and exploiting the high-quality learning ability of GGCNN2 architecture with the limited number of labelled dataset together with the learned latent embeddings, can be used as a de-facto training method which has been established and also validated in this paper through rigorous hardware experimentations using Baxter (Anukul) research robot.

Priya Shukla, Nilotpal Pramanik, Deepesh Mehta et al.

Our way of grasping objects is challenging for efficient, intelligent and optimal grasp by COBOTs. To streamline the process, here we use deep learning techniques to help robots learn to generate and execute appropriate grasps quickly. We developed a Generative Inception Neural Network (GI-NNet) model, capable of generating antipodal robotic grasps on seen as well as unseen objects. It is trained on Cornell Grasping Dataset (CGD) and attained 98.87% grasp pose accuracy for detecting both regular and irregular shaped objects from RGB-Depth (RGB-D) images while requiring only one third of the network trainable parameters as compared to the existing approaches. However, to attain this level of performance the model requires the entire 90% of the available labelled data of CGD keeping only 10% labelled data for testing which makes it vulnerable to poor generalization. Furthermore, getting sufficient and quality labelled dataset is becoming increasingly difficult keeping in pace with the requirement of gigantic networks. To address these issues, we attach our model as a decoder with a semi-supervised learning based architecture known as Vector Quantized Variational Auto Encoder (VQVAE), which works efficiently when trained both with the available labelled and unlabelled data. The proposed model, which we name as Representation based GI-NNet (RGI-NNet), has been trained with various splits of label data on CGD with as minimum as 10% labelled dataset together with latent embedding generated from VQVAE up to 50% labelled data with latent embedding obtained from VQVAE. The performance level, in terms of grasp pose accuracy of RGI-NNet, varies between 92.13% to 95.6% which is far better than several existing models trained with only labelled dataset. For the performance verification of both GI-NNet and RGI-NNet models, we use Anukul (Baxter) hardware cobot.

Priya Shukla, Hitesh Kumar, G. C. Nandi

Intelligent Object manipulation for grasping is a challenging problem for robots. Unlike robots, humans almost immediately know how to manipulate objects for grasping due to learning over the years. A grown woman can grasp objects more skilfully than a child because of learning skills developed over years, the absence of which in the present day robotic grasping compels it to perform well below the human object grasping benchmarks. In this paper we have taken up the challenge of developing learning based pose estimation by decomposing the problem into both position and orientation learning. More specifically, for grasp position estimation, we explore three different methods - a Genetic Algorithm (GA) based optimization method to minimize error between calculated image points and predicted end-effector (EE) position, a regression based method (RM) where collected data points of robot EE and image points have been regressed with a linear model, a PseudoInverse (PI) model which has been formulated in the form of a mapping matrix with robot EE position and image points for several observations. Further for grasp orientation learning, we develop a deep reinforcement learning (DRL) model which we name as Grasp Deep Q-Network (GDQN) and benchmarked our results with Modified VGG16 (MVGG16). Rigorous experimentations show that due to inherent capability of producing very high-quality solutions for optimization problems and search problems, GA based predictor performs much better than the other two models for position estimation. For orientation learning results indicate that off policy learning through GDQN outperforms MVGG16, since GDQN architecture is specially made suitable for the reinforcement learning. Based on our proposed architectures and algorithms, the robot is capable of grasping all rigid body objects having regular shapes.

Vijay Bhaskar Semwal, G. C. Nandi

Human can negotiate and recovers from Push up to certain extent. The push recovery capability grows with age (a child has poor push recovery than an adult) and it is based on learning. A wrestler, for example, has better push recovery than an ordinary man. However, the mechanism of reactive push recovery is not known to us. We tried to understand the human learning mechanism by conducting several experiments. The subjects for the experiments were selected both as right handed and left handed. Pushes were induced from the behind with close eyes to observe the motor action as well as with open eyes to observe learning based reactive behaviors. Important observations show that the left handed and right handed persons negotiate pushes differently (in opposite manner). The present research describes some details about the experiments and the analyses of the results mainly obtained from the joint angle variations (both for ankle and hip joints) as the manifestation of perturbation. After smoothening the captured data through higher order polynomials, we feed them to our model which was developed exploiting the physics of an inverted pendulum and configured it as a representative of the subjects in the Webot simulation framework available in our laboratory. In each cases the model also could recover from the push for the same rage of perturbation which proves the correctness of the model. Hence the model now can provide greater insight to push recovery mechanism and can be used for determining push recovery strategy for humanoid robots. The paper claimed the push recovery is software engineering problem rather than hardware.

Vijay Bhaskar Semwal, S. A. Katiyar, P. Chakraborty et al.

The present research as described in this paper tries to impart how imitation based learning for behavior-based programming can be used to teach the robot. This development is a big step in way to prove that push recovery is a software engineering problem and not hardware engineering problem. The walking algorithm used here aims to select a subset of push recovery problem i.e. disturbance from environment. We applied the physics at each joint of Halo with some degree of freedom. The proposed model, Halo is different from other models as previously developed model were inconsistent with data for different persons. This would lead to development of the generalized biped model in future and will bridge the gap between performance and inconsistency. In this paper the proposed model is applied to data of different persons. Accuracy of model, performance and result is measured using the behavior negotiation capability of model developed. In order to improve the performance, proposed model gives the freedom to handle each joint independently based on the belongingness value for each joint. The development can be considered as important development for future world of robotics. The accuracy of model is 70% in one go.