H2OFlow: Grounding Human-Object Affordances with 3D Generative Models and Dense Diffused Flows
This addresses a critical challenge in computer vision, robotics, and AI for enabling more natural human-object interactions, though it is incremental as it builds on existing 3D generative models and flow-based representations.
The paper tackles the problem of understanding 3D human-object interaction affordances, which often relies on costly hand-labeled datasets and limited contact-based analysis, by introducing H2OFlow, a framework that learns comprehensive affordances (contact, orientation, spatial occupancy) using only synthetic data from 3D generative models and demonstrates generalization to real-world objects, surpassing prior annotation-dependent methods.
Understanding how humans interact with the surrounding environment, and specifically reasoning about object interactions and affordances, is a critical challenge in computer vision, robotics, and AI. Current approaches often depend on labor-intensive, hand-labeled datasets capturing real-world or simulated human-object interaction (HOI) tasks, which are costly and time-consuming to produce. Furthermore, most existing methods for 3D affordance understanding are limited to contact-based analysis, neglecting other essential aspects of human-object interactions, such as orientation (\eg, humans might have a preferential orientation with respect certain objects, such as a TV) and spatial occupancy (\eg, humans are more likely to occupy certain regions around an object, like the front of a microwave rather than its back). To address these limitations, we introduce \emph{H2OFlow}, a novel framework that comprehensively learns 3D HOI affordances -- encompassing contact, orientation, and spatial occupancy -- using only synthetic data generated from 3D generative models. H2OFlow employs a dense 3D-flow-based representation, learned through a dense diffusion process operating on point clouds. This learned flow enables the discovery of rich 3D affordances without the need for human annotations. Through extensive quantitative and qualitative evaluations, we demonstrate that H2OFlow generalizes effectively to real-world objects and surpasses prior methods that rely on manual annotations or mesh-based representations in modeling 3D affordance.