Meta-Learning with Context-Agnostic Initialisations
This addresses a specific issue in meta-learning for few-shot problems, particularly in domains like character recognition and health monitoring, by making models more robust to irrelevant contextual variations, though it is incremental as it builds on existing meta-learning algorithms.
The paper tackled the problem of meta-learning being distracted by irrelevant context in training data, which harms performance on target tasks with novel contexts, by incorporating a context-adversarial component to produce context-agnostic initializations. The result showed improvements, such as a 4.3% average increase in accuracy on Omniglot character classification with unseen alphabets and a 30% reduction in mean square error on personalized energy expenditure predictions.
Meta-learning approaches have addressed few-shot problems by finding initialisations suited for fine-tuning to target tasks. Often there are additional properties within training data (which we refer to as context), not relevant to the target task, which act as a distractor to meta-learning, particularly when the target task contains examples from a novel context not seen during training. We address this oversight by incorporating a context-adversarial component into the meta-learning process. This produces an initialisation for fine-tuning to target which is both context-agnostic and task-generalised. We evaluate our approach on three commonly used meta-learning algorithms and two problems. We demonstrate our context-agnostic meta-learning improves results in each case. First, we report on Omniglot few-shot character classification, using alphabets as context. An average improvement of 4.3% is observed across methods and tasks when classifying characters from an unseen alphabet. Second, we evaluate on a dataset for personalised energy expenditure predictions from video, using participant knowledge as context. We demonstrate that context-agnostic meta-learning decreases the average mean square error by 30%.