Control in Stochastic Environment with Delays: A Model-based Reinforcement Learning Approach
This addresses control challenges in stochastic environments with delays for reinforcement learning applications, but appears incremental as it builds on existing methods with a focus on risk preference.
The paper tackles control problems in environments with delayed feedback by introducing a new reinforcement learning method that uses stochastic planning to embed risk preference in policy optimization, showing it can recover optimal policies for deterministic transitions and comparing it to prior methods on Atari games.
In this paper we are introducing a new reinforcement learning method for control problems in environments with delayed feedback. Specifically, our method employs stochastic planning, versus previous methods that used deterministic planning. This allows us to embed risk preference in the policy optimization problem. We show that this formulation can recover the optimal policy for problems with deterministic transitions. We contrast our policy with two prior methods from literature. We apply the methodology to simple tasks to understand its features. Then, we compare the performance of the methods in controlling multiple Atari games.