Understanding and Exploiting Object Interaction Landscapes
This work addresses the challenge of understanding object interactions for virtual and real-world agents, offering a novel representation that is agnostic to object types, which could be incremental in advancing scene understanding.
The paper tackles the problem of representing object interactions in a general way by introducing a new representation based on tracking particles and sensors, resulting in functional descriptors called interaction landscapes that capture object use and enable tasks like relating objects by function and establishing shape correspondences.
Interactions play a key role in understanding objects and scenes, for both virtual and real world agents. We introduce a new general representation for proximal interactions among physical objects that is agnostic to the type of objects or interaction involved. The representation is based on tracking particles on one of the participating objects and then observing them with sensors appropriately placed in the interaction volume or on the interaction surfaces. We show how to factorize these interaction descriptors and project them into a particular participating object so as to obtain a new functional descriptor for that object, its interaction landscape, capturing its observed use in a spatio-temporal framework. Interaction landscapes are independent of the particular interaction and capture subtle dynamic effects in how objects move and behave when in functional use. Our method relates objects based on their function, establishes correspondences between shapes based on functional key points and regions, and retrieves peer and partner objects with respect to an interaction.