GarNet: A Continuous Robot Vision Approach for Predicting Shapes and Visually Perceived Weights of Garments
This work addresses the challenge of robot manipulation of garments, which is incremental by building on existing vision approaches with a continuous observation method.
The paper tackles the problem of predicting garment shape and visually perceived weight from robot vision, achieving 92% accuracy for shape classification and 95.5% for weight prediction, with a 21% improvement over state-of-the-art methods.
We present a Garment Similarity Network (GarNet) that learns geometric and physical similarities between known garments by continuously observing a garment while a robot picks it up from a table. The aim is to capture and encode geometric and physical characteristics of a garment into a manifold where a decision can be carried out, such as predicting the garment's shape class and its visually perceived weight. Our approach features an early stop strategy, which means that GarNet does not need to observe a garment being picked up from a crumpled to a hanging state to make a prediction. In our experiments, we find that GarNet achieves prediction accuracies of 92% for shape classification and 95.5% for predicting weights and advances state-of-art approaches by 21% for shape classification.