Classical Policy Gradient: Preserving Bellman's Principle of Optimality

This work addresses a theoretical gap for researchers in reinforcement learning, but appears incremental as it modifies an existing framework without broad empirical validation.

The authors tackled the problem of aligning policy gradient methods with Bellman's principle of optimality in finite-horizon episodic Markov decision processes by proposing a new objective function and deriving its gradient expression.