Off-Policy Shaping Ensembles in Reinforcement Learning
This work addresses performance enhancement in reinforcement learning for practitioners, but it is incremental as it builds on existing gradient temporal-difference methods and ensemble techniques.
The paper tackled the problem of improving reinforcement learning performance by proposing an ensemble of policies related through potential-based shaping rewards, and the result showed that the combination policy outperformed individual ensemble policies in empirical evaluations.
Recent advances of gradient temporal-difference methods allow to learn off-policy multiple value functions in parallel with- out sacrificing convergence guarantees or computational efficiency. This opens up new possibilities for sound ensemble techniques in reinforcement learning. In this work we propose learning an ensemble of policies related through potential-based shaping rewards. The ensemble induces a combination policy by using a voting mechanism on its components. Learning happens in real time, and we empirically show the combination policy to outperform the individual policies of the ensemble.