Where2Start: Leveraging initial States for Robust and Sample-Efficient Reinforcement Learning
This addresses sample inefficiency for reinforcement learning practitioners, offering an incremental improvement that can be combined with existing algorithms.
The paper tackles the problem of sample inefficiency in reinforcement learning by proposing the Where2Start algorithm, which selects initial states to increase agent instability and retrieve more informative trajectories, resulting in up to 8 times improvement in sample efficiency.
The reinforcement learning algorithms that focus on how to compute the gradient and choose next actions, are effectively improved the performance of the agents. However, these algorithms are environment-agnostic. This means that the algorithms did not use the knowledge that has been captured by trajectory. This poses that the algorithms should sample many trajectories to train the model. By considering the essence of environment and how much the agent learn from each scenario in that environment, the strategy of the learning procedure can be changed. The strategy retrieves more informative trajectories, so the agent can learn with fewer trajectory sample. We propose Where2Start algorithm that selects the initial state so that the agent has more instability in vicinity of that state. We show that this kind of selection decreases number of trajectories that should be sampled that the agent reach to acceptable reward. Our experiments shows that Where2Start can improve sample efficiency up to 8 times. Also Where2Start can combined with most of state-of-the-art algorithms and improve that robustness and sample efficiency significantly.