A Hybrid PAC Reinforcement Learning Algorithm
This work addresses sample efficiency for reinforcement learning practitioners, but it is incremental as it builds on and combines existing methods.
The paper tackles the problem of improving sample efficiency in reinforcement learning by proposing a hybrid PAC algorithm called Dyna-Delayed Q-learning (DDQ), which combines model-free and model-based approaches and outperforms existing methods in most cases, with numerical results supporting its efficiency.
This paper offers a new hybrid probably approximately correct (PAC) reinforcement learning (RL) algorithm for Markov decision processes (MDPs) that intelligently maintains favorable features of its parents. The designed algorithm, referred to as the Dyna-Delayed Q-learning (DDQ) algorithm, combines model-free and model-based learning approaches while outperforming both in most cases. The paper includes a PAC analysis of the DDQ algorithm and a derivation of its sample complexity. Numerical results are provided to support the claim regarding the new algorithm's sample efficiency compared to its parents as well as the best known model-free and model-based algorithms in application.