FAST: Similarity-based Knowledge Transfer for Efficient Policy Learning
This work addresses the challenge of reducing computational costs and improving agent adaptation in evolving domains like game development, though it is incremental in its approach to transfer learning.
The paper tackles the problem of inefficient knowledge transfer in reinforcement learning by proposing FAST, a framework that uses visual and textual embeddings to estimate task similarity and select source policies, achieving competitive final performance with significantly fewer training steps across multiple racing tracks.
Transfer Learning (TL) offers the potential to accelerate learning by transferring knowledge across tasks. However, it faces critical challenges such as negative transfer, domain adaptation and inefficiency in selecting solid source policies. These issues often represent critical problems in evolving domains, i.e. game development, where scenarios transform and agents must adapt. The continuous release of new agents is costly and inefficient. In this work we challenge the key issues in TL to improve knowledge transfer, agents performance across tasks and reduce computational costs. The proposed methodology, called FAST - Framework for Adaptive Similarity-based Transfer, leverages visual frames and textual descriptions to create a latent representation of tasks dynamics, that is exploited to estimate similarity between environments. The similarity scores guides our method in choosing candidate policies from which transfer abilities to simplify learning of novel tasks. Experimental results, over multiple racing tracks, demonstrate that FAST achieves competitive final performance compared to learning-from-scratch methods while requiring significantly less training steps. These findings highlight the potential of embedding-driven task similarity estimations.