Exploiting Symmetry in Dynamics for Model-Based Reinforcement Learning with Asymmetric Rewards
This work extends symmetry techniques to a broader class of reinforcement learning problems, though it appears incremental as it builds on existing symmetry methods.
The paper tackles the problem of model-based reinforcement learning in environments where dynamics have symmetry but rewards do not, by using Cartan's moving frame method to learn symmetric dynamics, resulting in a more accurate dynamical model as shown in numerical experiments.
Recent work in reinforcement learning has leveraged symmetries in the model to improve sample efficiency in training a policy. A commonly used simplifying assumption is that the dynamics and reward both exhibit the same symmetry; however, in many real-world environments, the dynamical model exhibits symmetry independent of the reward model. In this paper, we assume only the dynamics exhibit symmetry, extending the scope of problems in reinforcement learning and learning in control theory to which symmetry techniques can be applied. We use Cartan's moving frame method to introduce a technique for learning dynamics that, by construction, exhibit specified symmetries. Numerical experiments demonstrate that the proposed method learns a more accurate dynamical model