IAAO: Interactive Affordance Learning for Articulated Objects in 3D Environments
This addresses the challenge of robotic manipulation of complex objects in real-world settings, representing a domain-specific incremental advance.
The paper tackles the problem of enabling intelligent agents to understand and manipulate articulated objects in 3D environments by building explicit 3D models through interaction, achieving effective results as demonstrated in experiments.
This work presents IAAO, a novel framework that builds an explicit 3D model for intelligent agents to gain understanding of articulated objects in their environment through interaction. Unlike prior methods that rely on task-specific networks and assumptions about movable parts, our IAAO leverages large foundation models to estimate interactive affordances and part articulations in three stages. We first build hierarchical features and label fields for each object state using 3D Gaussian Splatting (3DGS) by distilling mask features and view-consistent labels from multi-view images. We then perform object- and part-level queries on the 3D Gaussian primitives to identify static and articulated elements, estimating global transformations and local articulation parameters along with affordances. Finally, scenes from different states are merged and refined based on the estimated transformations, enabling robust affordance-based interaction and manipulation of objects. Experimental results demonstrate the effectiveness of our method.