GPG: Generalized Policy Gradient Theorem for Transformer-based Policies

This work addresses policy optimization challenges for researchers and practitioners using Transformer-based models in reinforcement learning, though it appears incremental as it generalizes existing theorems rather than introducing a completely new paradigm.

The paper tackles the problem of policy optimization for Transformer-based policies by introducing the Generalized Policy Gradient (GPG) Theorem, which unifies existing methods like the standard Policy Gradient Theorem and GRPO as special cases, and demonstrates its practical applications in training Large Language Models (LLMs) for efficient policy optimization.