Semantics in Robotics: Environmental Data Can't Yield Conventions of Human Behaviour
This addresses a foundational problem in robotics and AI for researchers by clarifying the limitations of semantics, though it is incremental as it builds on existing critiques.
The paper argues that semantics in robotics cannot be derived from environmental data alone, but must instead be based on conventions of human behavior, such as labels and affordances, which require understanding beyond current AI capabilities.
The word semantics, in robotics and AI, has no canonical definition. It usually serves to denote additional data provided to autonomous agents to aid HRI. Most researchers seem, implicitly, to understand that such data cannot simply be extracted from environmental data. I try to make explicit why this is so and argue that so-called semantics are best understood as data comprised of conventions of human behaviour. This includes labels, most obviously, but also places, ontologies, and affordances. Object affordances are especially problematic because they require not only semantics that are not in the environmental data (conventions of object use) but also an understanding of physics and object combinations that would, if achieved, constitute artificial superintelligence.