The Natural Language of Actions
This addresses the challenge of action representation in reinforcement learning for domains with large or complex action spaces, though it appears incremental as it builds on existing embedding methods.
The authors tackled the problem of representing actions in reinforcement learning by introducing Act2Vec, a framework that learns context-based action representations from demonstrations, resulting in improved performance across domains like drawing, navigation, and StarCraft II.
We introduce Act2Vec, a general framework for learning context-based action representation for Reinforcement Learning. Representing actions in a vector space help reinforcement learning algorithms achieve better performance by grouping similar actions and utilizing relations between different actions. We show how prior knowledge of an environment can be extracted from demonstrations and injected into action vector representations that encode natural compatible behavior. We then use these for augmenting state representations as well as improving function approximation of Q-values. We visualize and test action embeddings in three domains including a drawing task, a high dimensional navigation task, and the large action space domain of StarCraft II.