Investigating the Impact of Action Representations in Policy Gradient Algorithms
This work addresses the problem of understanding performance variations in RL algorithms for researchers, but it is incremental as it focuses on analysis techniques rather than proposing new methods.
The paper investigates how different action representations affect the learning performance of policy gradient algorithms on popular RL benchmarks, finding that these representations can significantly influence performance due to changes in optimization landscape complexity.
Reinforcement learning~(RL) is a versatile framework for learning to solve complex real-world tasks. However, influences on the learning performance of RL algorithms are often poorly understood in practice. We discuss different analysis techniques and assess their effectiveness for investigating the impact of action representations in RL. Our experiments demonstrate that the action representation can significantly influence the learning performance on popular RL benchmark tasks. The analysis results indicate that some of the performance differences can be attributed to changes in the complexity of the optimization landscape. Finally, we discuss open challenges of analysis techniques for RL algorithms.