Learning to Transfer Learn: Reinforcement Learning-Based Selection for Adaptive Transfer Learning
This work addresses adaptive transfer learning for scenarios with limited target data or mismatched labels, representing an incremental improvement over existing methods.
The paper tackles the problem of improving performance on a target dataset by adaptively extracting information from a source dataset, resulting in L2TL outperforming baselines on eight datasets, with particularly large benefits for small-scale target datasets and significant label mismatches.
We propose a novel adaptive transfer learning framework, learning to transfer learn (L2TL), to improve performance on a target dataset by careful extraction of the related information from a source dataset. Our framework considers cooperative optimization of shared weights between models for source and target tasks, and adjusts the constituent loss weights adaptively. The adaptation of the weights is based on a reinforcement learning (RL) selection policy, guided with a performance metric on the target validation set. We demonstrate that L2TL outperforms fine-tuning baselines and other adaptive transfer learning methods on eight datasets. In the regimes of small-scale target datasets and significant label mismatch between source and target datasets, L2TL shows particularly large benefits.