R3L: Relative Representations for Reinforcement Learning
This addresses the issue of model adaptability in visual RL for AI systems, though it is incremental as it builds on existing representation learning frameworks.
The paper tackles the problem of visual reinforcement learning agents failing when faced with new visual or task variations, requiring retraining. It adapts relative representations to combine agent components zero-shot, enabling effective handling of novel visual-task pairs and reducing retraining needs.
Visual Reinforcement Learning is a popular and powerful framework that takes full advantage of the Deep Learning breakthrough. It is known that variations in input domains (e.g., different panorama colors due to seasonal changes) or task domains (e.g., altering the target speed of a car) can disrupt agent performance, necessitating new training for each variation. Recent advancements in the field of representation learning have demonstrated the possibility of combining components from different neural networks to create new models in a zero-shot fashion. In this paper, we build upon relative representations, a framework that maps encoder embeddings to a universal space. We adapt this framework to the Visual Reinforcement Learning setting, allowing to combine agents components to create new agents capable of effectively handling novel visual-task pairs not encountered during training. Our findings highlight the potential for model reuse, significantly reducing the need for retraining and, consequently, the time and computational resources required.