Where is My Stuff? An Interactive System for Spatial Relations
This addresses the problem of enabling robots to understand and communicate spatial relations for improved human-robot interaction, though it is currently static and incremental in scope.
The paper presents an interactive system that detects and tracks objects and agents at 30 frames per second using an RGBD camera, computes spatial relations like 'in' and 'on', and communicates them to users via speech, with a focus on grounding language for human-robot interaction.
In this paper we present a system that detects and tracks objects and agents, computes spatial relations, and communicates those relations to the user using speech. Our system is able to detect multiple objects and agents at 30 frames per second using a RGBD camera. It is able to extract the spatial relations in, on, next to, near, and belongs to, and communicate these relations using natural language. The notion of belonging is particularly important for Human-Robot Interaction since it allows the robot ground the language and reason about the right objects. Although our system is currently static and targeted to a fixed location in a room, we are planning to port it to a mobile robot thus allowing it explore the environment and create a spatial knowledge base.