Model Reuse with Reduced Kernel Mean Embedding Specification
This addresses the challenge of efficient model reuse for machine learning practitioners, though it appears incremental as it builds on existing kernel mean embedding methods.
The paper tackles the problem of reusing pre-trained models from a public pool for new tasks without accessing their raw training data, by introducing a two-phase framework using reduced kernel mean embedding (RKME) specifications to measure task relatedness, with theoretical and experimental validation showing effectiveness.
Given a publicly available pool of machine learning models constructed for various tasks, when a user plans to build a model for her own machine learning application, is it possible to build upon models in the pool such that the previous efforts on these existing models can be reused rather than starting from scratch? Here, a grand challenge is how to find models that are helpful for the current application, without accessing the raw training data for the models in the pool. In this paper, we present a two-phase framework. In the upload phase, when a model is uploading into the pool, we construct a reduced kernel mean embedding (RKME) as a specification for the model. Then in the deployment phase, the relatedness of the current task and pre-trained models will be measured based on the value of the RKME specification. Theoretical results and extensive experiments validate the effectiveness of our approach.