Learning Discrete Abstractions for Visual Rearrangement Tasks Using Vision-Guided Graph Coloring
This addresses the scalability and applicability of planning frameworks in robotics by reducing the need for hand-engineered abstractions, though it is incremental as it builds on existing hierarchical reasoning concepts.
The paper tackles the problem of automating the discovery of discrete abstractions from visual data for rearrangement tasks in robotics, showing that their method consistently identifies meaningful abstractions that facilitate effective planning and outperform existing approaches.
Learning abstractions directly from data is a core challenge in robotics. Humans naturally operate at an abstract level, reasoning over high-level subgoals while delegating execution to low-level motor skills -- an ability that enables efficient problem solving in complex environments. In robotics, abstractions and hierarchical reasoning have long been central to planning, yet they are typically hand-engineered, demanding significant human effort and limiting scalability. Automating the discovery of useful abstractions directly from visual data would make planning frameworks more scalable and more applicable to real-world robotic domains. In this work, we focus on rearrangement tasks where the state is represented with raw images, and propose a method to induce discrete, graph-structured abstractions by combining structural constraints with an attention-guided visual distance. Our approach leverages the inherent bipartite structure of rearrangement problems, integrating structural constraints and visual embeddings into a unified framework. This enables the autonomous discovery of abstractions from vision alone, which can subsequently support high-level planning. We evaluate our method on two rearrangement tasks in simulation and show that it consistently identifies meaningful abstractions that facilitate effective planning and outperform existing approaches.