Learning Intermediate Features of Object Affordances with a Convolutional Neural Network
This work addresses the challenge of explicating neural mechanisms for object-action associations in the visuomotor pathway, which is incremental as it builds on existing deep learning methods to model affordances.
The researchers tackled the problem of understanding object affordances by training a convolutional neural network to recognize affordances from images and learn their underlying features, aiming to test these features against human neural data.
Our ability to interact with the world around us relies on being able to infer what actions objects afford -- often referred to as affordances. The neural mechanisms of object-action associations are realized in the visuomotor pathway where information about both visual properties and actions is integrated into common representations. However, explicating these mechanisms is particularly challenging in the case of affordances because there is hardly any one-to-one mapping between visual features and inferred actions. To better understand the nature of affordances, we trained a deep convolutional neural network (CNN) to recognize affordances from images and to learn the underlying features or the dimensionality of affordances. Such features form an underlying compositional structure for the general representation of affordances which can then be tested against human neural data. We view this representational analysis as the first step towards a more formal account of how humans perceive and interact with the environment.