Memory Allocation in Resource-Constrained Reinforcement Learning
This addresses a resource limitation issue for reinforcement learning systems, but it is incremental as it applies existing methods to constrained scenarios.
The paper tackles the problem of how memory constraints affect reinforcement learning agents by studying the trade-off between allocating memory for world modeling versus planning, finding that different allocations impact performance in episodic and continual learning settings with MCTS- and DQN-based algorithms.
Resource constraints can fundamentally change both learning and decision-making. We explore how memory constraints influence an agent's performance when navigating unknown environments using standard reinforcement learning algorithms. Specifically, memory-constrained agents face a dilemma: how much of their limited memory should be allocated to each of the agent's internal processes, such as estimating a world model, as opposed to forming a plan using that model? We study this dilemma in MCTS- and DQN-based algorithms and examine how different allocations of memory impact performance in episodic and continual learning settings.