Meta-Learning with Heterogeneous Tasks
This addresses a limitation in meta-learning for real-world applications where tasks are not equally important, though it is an incremental improvement over existing methods.
The paper tackles the problem of meta-learning with heterogeneous tasks, where tasks vary in difficulty or noise, by introducing HeTRoM, a method that uses rank-based objectives to prevent easy tasks from dominating learning, resulting in improved flexibility and overall performance in diverse task settings.
Meta-learning is a general approach to equip machine learning models with the ability to handle few-shot scenarios when dealing with many tasks. Most existing meta-learning methods work based on the assumption that all tasks are of equal importance. However, real-world applications often present heterogeneous tasks characterized by varying difficulty levels, noise in training samples, or being distinctively different from most other tasks. In this paper, we introduce a novel meta-learning method designed to effectively manage such heterogeneous tasks by employing rank-based task-level learning objectives, Heterogeneous Tasks Robust Meta-learning (HeTRoM). HeTRoM is proficient in handling heterogeneous tasks, and it prevents easy tasks from overwhelming the meta-learner. The approach allows for an efficient iterative optimization algorithm based on bi-level optimization, which is then improved by integrating statistical guidance. Our experimental results demonstrate that our method provides flexibility, enabling users to adapt to diverse task settings and enhancing the meta-learner's overall performance.