Precise Affordance Annotation for Egocentric Action Video Datasets
This work addresses annotation issues for researchers in human-object interaction and robotics, but it is incremental as it refines existing dataset annotations rather than introducing a new method or paradigm.
The paper tackles the problem of imprecise affordance annotation in egocentric action video datasets by proposing a new annotation scheme that separates affordance from object functionality and goal-related actions, and applies it to the EPIC-KITCHENS dataset, showing that models trained with these annotations can distinguish affordance and mechanical actions.
Object affordance is an important concept in human-object interaction, providing information on action possibilities based on human motor capacity and objects' physical property thus benefiting tasks such as action anticipation and robot imitation learning. However, existing datasets often: 1) mix up affordance with object functionality; 2) confuse affordance with goal-related action; and 3) ignore human motor capacity. This paper proposes an efficient annotation scheme to address these issues by combining goal-irrelevant motor actions and grasp types as affordance labels and introducing the concept of mechanical action to represent the action possibilities between two objects. We provide new annotations by applying this scheme to the EPIC-KITCHENS dataset and test our annotation with tasks such as affordance recognition. We qualitatively verify that models trained with our annotation can distinguish affordance and mechanical actions.