What can you do with a rock? Affordance extraction via word embeddings
This work addresses the challenge of efficient action selection in reinforcement learning agents, though it is incremental as it applies an existing word embedding technique to a new domain.
The paper tackles the problem of affordance detection for autonomous agents in large action spaces by using word embeddings from Wikipedia to extract affordances, showing that this method improves agent performance and reduces total learning steps while making action selections more human-like.
Autonomous agents must often detect affordances: the set of behaviors enabled by a situation. Affordance detection is particularly helpful in domains with large action spaces, allowing the agent to prune its search space by avoiding futile behaviors. This paper presents a method for affordance extraction via word embeddings trained on a Wikipedia corpus. The resulting word vectors are treated as a common knowledge database which can be queried using linear algebra. We apply this method to a reinforcement learning agent in a text-only environment and show that affordance-based action selection improves performance most of the time. Our method increases the computational complexity of each learning step but significantly reduces the total number of steps needed. In addition, the agent's action selections begin to resemble those a human would choose.