A General Family of Robust Stochastic Operators for Reinforcement Learning
This work addresses robustness issues in reinforcement learning for practitioners, though it appears incremental as it builds on existing operator frameworks.
The authors tackled the problem of approximation and estimation errors in reinforcement learning by introducing a new family of robust stochastic operators, which preserved optimality and increased the action gap, and empirically outperformed classical and recent operators.
We consider a new family of operators for reinforcement learning with the goal of alleviating the negative effects and becoming more robust to approximation or estimation errors. Various theoretical results are established, which include showing on a sample path basis that our family of operators preserve optimality and increase the action gap. Our empirical results illustrate the strong benefits of our family of operators, significantly outperforming the classical Bellman operator and recently proposed operators.