Policy gradient methods for ordinal policies
This work addresses a practical challenge in reinforcement learning for domains where actions have inherent order, offering an incremental improvement over existing methods.
The paper tackled the problem of standard softmax policies in reinforcement learning failing to capture order relationships between actions, proposing a novel ordinal regression-based policy parametrization that demonstrated effectiveness in real-world industrial applications and competitive performance in continuous action tasks.
In reinforcement learning, the softmax parametrization is the standard approach for policies over discrete action spaces. However, it fails to capture the order relationship between actions. Motivated by a real-world industrial problem, we propose a novel policy parametrization based on ordinal regression models adapted to the reinforcement learning setting. Our approach addresses practical challenges, and numerical experiments demonstrate its effectiveness in real applications and in continuous action tasks, where discretizing the action space and applying the ordinal policy yields competitive performance.