Diverse Exploration via Conjugate Policies for Policy Gradient Methods
This addresses exploration inefficiencies in reinforcement learning, particularly for policy gradient methods, offering a novel approach to enhance performance, though it appears incremental as it builds on existing conjugate gradient techniques.
The paper tackles the challenge of effective exploration in policy gradient methods by proposing diverse exploration via conjugate policies, which learns and deploys a set of conjugate policies generated from conjugate gradient descent, showing improved policy performance and advantages over random perturbations in empirical results.
We address the challenge of effective exploration while maintaining good performance in policy gradient methods. As a solution, we propose diverse exploration (DE) via conjugate policies. DE learns and deploys a set of conjugate policies which can be conveniently generated as a byproduct of conjugate gradient descent. We provide both theoretical and empirical results showing the effectiveness of DE at achieving exploration, improving policy performance, and the advantage of DE over exploration by random policy perturbations.