SETrLUSI: Stochastic Ensemble Multi-Source Transfer Learning Using Statistical Invariant
This work addresses multi-source transfer learning for machine learning practitioners, offering an incremental improvement in efficiency and stability.
The paper tackles the problem of multi-source transfer learning by introducing an ensemble framework that extracts and integrates diverse knowledge from source and target domains using Statistical Invariant, resulting in improved convergence and lower time cost compared to related methods.
In transfer learning, a source domain often carries diverse knowledge, and different domains usually emphasize different types of knowledge. Different from handling only a single type of knowledge from all domains in traditional transfer learning methods, we introduce an ensemble learning framework with a weak mode of convergence in the form of Statistical Invariant (SI) for multi-source transfer learning, formulated as Stochastic Ensemble Multi-Source Transfer Learning Using Statistical Invariant (SETrLUSI). The proposed SI extracts and integrates various types of knowledge from both source and target domains, which not only effectively utilizes diverse knowledge but also accelerates the convergence process. Further, SETrLUSI incorporates stochastic SI selection, proportional source domain sampling, and target domain bootstrapping, which improves training efficiency while enhancing model stability. Experiments show that SETrLUSI has good convergence and outperforms related methods with a lower time cost.