Reinforcement Learning with Lie Group Orientations for Robotics
This addresses orientation representation issues in robotics learning pipelines, though it appears incremental as a modification to existing methods.
The paper tackles the problem of mathematically incorrect orientation handling in reinforcement learning for robotics by proposing a simple network modification that adheres to Lie group structure, achieving significantly better performance than common representations in tasks like orientation control and pick-and-place.
Handling orientations of robots and objects is a crucial aspect of many applications. Yet, ever so often, there is a lack of mathematical correctness when dealing with orientations, especially in learning pipelines involving, for example, artificial neural networks. In this paper, we investigate reinforcement learning with orientations and propose a simple modification of the network's input and output that adheres to the Lie group structure of orientations. As a result, we obtain an easy and efficient implementation that is directly usable with existing learning libraries and achieves significantly better performance than other common orientation representations. We briefly introduce Lie theory specifically for orientations in robotics to motivate and outline our approach. Subsequently, a thorough empirical evaluation of different combinations of orientation representations for states and actions demonstrates the superior performance of our proposed approach in different scenarios, including: direct orientation control, end effector orientation control, and pick-and-place tasks.