LangPert: Detecting and Handling Task-level Perturbations for Robust Object Rearrangement
This addresses robustness issues in tabletop object rearrangement for robotics applications, representing an incremental improvement through a novel integration of existing components.
The paper tackles the problem of task-level perturbations (unexpected object changes) disrupting object rearrangement tasks by introducing LangPert, a language-based framework that detects and mitigates these perturbations using visual language models and hierarchical reasoning. The result shows LangPert handles diverse perturbations more effectively than baselines, achieving higher task completion rates and improved execution efficiency.
Task execution for object rearrangement could be challenged by Task-Level Perturbations (TLP), i.e., unexpected object additions, removals, and displacements that can disrupt underlying visual policies and fundamentally compromise task feasibility and progress. To address these challenges, we present LangPert, a language-based framework designed to detect and mitigate TLP situations in tabletop rearrangement tasks. LangPert integrates a Visual Language Model (VLM) to comprehensively monitor policy's skill execution and environmental TLP, while leveraging the Hierarchical Chain-of-Thought (HCoT) reasoning mechanism to enhance the Large Language Model (LLM)'s contextual understanding and generate adaptive, corrective skill-execution plans. Our experimental results demonstrate that LangPert handles diverse TLP situations more effectively than baseline methods, achieving higher task completion rates, improved execution efficiency, and potential generalization to unseen scenarios.