Learning State Abstractions for Transfer in Continuous Control
This addresses the challenge of sample efficiency and transfer learning in reinforcement learning for continuous control, though it is incremental as it builds on existing abstraction methods.
The paper tackles the problem of enabling simple tabular Q-Learning to solve continuous control tasks by learning state abstractions that discretize the state-space, and it shows that transferring these abstractions to unseen tasks allows efficient learning with bounded value loss.
Can simple algorithms with a good representation solve challenging reinforcement learning problems? In this work, we answer this question in the affirmative, where we take "simple learning algorithm" to be tabular Q-Learning, the "good representations" to be a learned state abstraction, and "challenging problems" to be continuous control tasks. Our main contribution is a learning algorithm that abstracts a continuous state-space into a discrete one. We transfer this learned representation to unseen problems to enable effective learning. We provide theory showing that learned abstractions maintain a bounded value loss, and we report experiments showing that the abstractions empower tabular Q-Learning to learn efficiently in unseen tasks.