Tool Substitution with Shape and Material Reasoning Using Dual Neural Networks
This addresses tool substitution for robotics or AI systems, but it is incremental as it builds on prior work by adding material reasoning.
The paper tackles the problem of identifying substitute tools for tasks by combining shape and material reasoning using dual neural networks, achieving improved performance on a test set of 30 real-world objects.
This paper explores the problem of tool substitution, namely, identifying substitute tools for performing a task from a given set of candidate tools. We introduce a novel approach to tool substitution, that unlike prior work in the area, combines both shape and material reasoning to effectively identify substitute tools. Our approach combines the use of visual and spectral reasoning using dual neural networks. It takes as input, the desired action to be performed, and outputs a ranking of the available candidate tools based on their suitability for performing the action. Our results on a test set of 30 real-world objects show that our approach is able to effectively match shape and material similarities, with improved tool substitution performance when combining both.