Model Recycling Framework for Multi-Source Data-Free Supervised Transfer Learning
This addresses data privacy and accessibility issues in transfer learning for scenarios where source data cannot be retrieved, though it is incremental as it builds on existing transfer learning concepts.
The paper tackles the problem of source-free transfer learning where source data is unavailable, proposing a model recycling framework that enables parameter-efficient training by reusing subsets of related source models in both white-box and black-box settings, allowing Model as a Service providers to build efficient pre-trained model libraries.
Increasing concerns for data privacy and other difficulties associated with retrieving source data for model training have created the need for source-free transfer learning, in which one only has access to pre-trained models instead of data from the original source domains. This setting introduces many challenges, as many existing transfer learning methods typically rely on access to source data, which limits their direct applicability to scenarios where source data is unavailable. Further, practical concerns make it more difficult, for instance efficiently selecting models for transfer without information on source data, and transferring without full access to the source models. So motivated, we propose a model recycling framework for parameter-efficient training of models that identifies subsets of related source models to reuse in both white-box and black-box settings. Consequently, our framework makes it possible for Model as a Service (MaaS) providers to build libraries of efficient pre-trained models, thus creating an opportunity for multi-source data-free supervised transfer learning.