SPOTS: Stable Placement of Objects with Reasoning in Semi-Autonomous Teleoperation Systems
This addresses the problem of reliable object placement in semi-autonomous robotics for applications like teleoperation, though it appears incremental as it builds on existing simulation and language model techniques.
The paper tackles the under-explored 'place' task in robotics pick-and-place by developing a method that ensures stable and contextually reasonable object placements in teleoperation systems, showing it greatly increases physical plausibility and contextual soundness in evaluations across simulation and real-world environments.
Pick-and-place is one of the fundamental tasks in robotics research. However, the attention has been mostly focused on the ``pick'' task, leaving the ``place'' task relatively unexplored. In this paper, we address the problem of placing objects in the context of a teleoperation framework. Particularly, we focus on two aspects of the place task: stability robustness and contextual reasonableness of object placements. Our proposed method combines simulation-driven physical stability verification via real-to-sim and the semantic reasoning capability of large language models. In other words, given place context information (e.g., user preferences, object to place, and current scene information), our proposed method outputs a probability distribution over the possible placement candidates, considering the robustness and reasonableness of the place task. Our proposed method is extensively evaluated in two simulation and one real world environments and we show that our method can greatly increase the physical plausibility of the placement as well as contextual soundness while considering user preferences.