Multi-Objective Meta Learning
This addresses inefficiencies in multi-objective meta learning for researchers and practitioners, offering a more efficient alternative to existing methods, though it appears incremental as it builds on gradient-based techniques.
The paper tackles the problem of meta learning with multiple conflicting objectives by proposing a unified gradient-based framework (MOML) and the first gradient-based optimization algorithm for Multi-Objective Bi-Level optimization, proving convergence and showing effectiveness in tasks like few-shot learning and neural architecture search.
Meta learning with multiple objectives can be formulated as a Multi-Objective Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is to solve several possible conflicting targets for the meta learner. However, existing studies either apply an inefficient evolutionary algorithm or linearly combine multiple objectives as a single-objective problem with the need to tune combination weights. In this paper, we propose a unified gradient-based Multi-Objective Meta Learning (MOML) framework and devise the first gradient-based optimization algorithm to solve the MOBLP by alternatively solving the lower-level and upper-level subproblems via the gradient descent method and the gradient-based multi-objective optimization method, respectively. Theoretically, we prove the convergence properties of the proposed gradient-based optimization algorithm. Empirically, we show the effectiveness of the proposed MOML framework in several meta learning problems, including few-shot learning, neural architecture search, domain adaptation, and multi-task learning.