Time Adaptive Reinforcement Learning
This work addresses the challenge of time adaptability in RL for scenarios where task execution time limits change, offering a general mechanism that can be integrated with existing methods, though it appears incremental as it builds on standard RL frameworks.
The paper tackles the problem of reinforcement learning agents being fixed to specific tasks and unable to adapt to different time restrictions, such as varying time limits for task completion, by introducing Time Adaptive Markov Decision Processes and two model-free algorithms that enable zero-shot adaptation between different time constraints.
Reinforcement learning (RL) allows to solve complex tasks such as Go often with a stronger performance than humans. However, the learned behaviors are usually fixed to specific tasks and unable to adapt to different contexts. Here we consider the case of adapting RL agents to different time restrictions, such as finishing a task with a given time limit that might change from one task execution to the next. We define such problems as Time Adaptive Markov Decision Processes and introduce two model-free, value-based algorithms: the Independent Gamma-Ensemble and the n-Step Ensemble. In difference to classical approaches, they allow a zero-shot adaptation between different time restrictions. The proposed approaches represent general mechanisms to handle time adaptive tasks making them compatible with many existing RL methods, algorithms, and scenarios.