Chameleon: Learning Model Initializations Across Tasks With Different Schemas
This addresses the limitation in meta-learning where tasks must share the same schema, enabling broader applicability across diverse datasets, though it is incremental by extending existing methods to handle schema variability.
The paper tackles the problem of meta-learning parameter initializations across tasks with different schemas, such as varying numbers of predictors, and proposes Chameleon to align these schemas to a common representation, demonstrating its success on 23 datasets from the OpenML-CC18 benchmark as the first cross-dataset few-shot classification approach for unstructured data.
Parametric models, and particularly neural networks, require weight initialization as a starting point for gradient-based optimization. Recent work shows that a specific initial parameter set can be learned from a population of supervised learning tasks. Using this initial parameter set enables a fast convergence for unseen classes even when only a handful of instances is available (model-agnostic meta-learning). Currently, methods for learning model initializations are limited to a population of tasks sharing the same schema, i.e., the same number, order, type, and semantics of predictor and target variables. In this paper, we address the problem of meta-learning parameter initialization across tasks with different schemas, i.e., if the number of predictors varies across tasks, while they still share some variables. We propose Chameleon, a model that learns to align different predictor schemas to a common representation. In experiments on 23 datasets of the OpenML-CC18 benchmark, we show that Chameleon can successfully learn parameter initializations across tasks with different schemas, presenting, to the best of our knowledge, the first cross-dataset few-shot classification approach for unstructured data.