Shaping Proto-Value Functions via Rewards
This work addresses a specific challenge in reinforcement learning for agents in environments with symmetrical states but asymmetrical rewards, offering an incremental improvement over existing methods.
The paper tackled the problem of improving reinforcement learning by combining reward shaping with proto-value functions to create reward-dependent proto-value functions (RPVFs), which better capture asymmetrical rewards in symmetrical state spaces, resulting in enhanced learning performance compared to using either method alone.
In this paper, we combine task-dependent reward shaping and task-independent proto-value functions to obtain reward dependent proto-value functions (RPVFs). In constructing the RPVFs we are making use of the immediate rewards which are available during the sampling phase but are not used in the PVF construction. We show via experiments that learning with an RPVF based representation is better than learning with just reward shaping or PVFs. In particular, when the state space is symmetrical and the rewards are asymmetrical, the RPVF capture the asymmetry better than the PVFs.