Meta-Learning Loss Functions for Deep Neural Networks
This addresses the challenge of reducing data requirements for AI systems, potentially benefiting fields with scarce data, though it appears incremental as it builds on existing meta-learning approaches.
The paper tackles the problem of improving deep neural network performance by meta-learning loss functions, which are crucial for defining learning objectives, aiming to enhance efficiency and effectiveness in learning from limited data.
Humans can often quickly and efficiently solve complex new learning tasks given only a small set of examples. In contrast, modern artificially intelligent systems often require thousands or millions of observations in order to solve even the most basic tasks. Meta-learning aims to resolve this issue by leveraging past experiences from similar learning tasks to embed the appropriate inductive biases into the learning system. Historically methods for meta-learning components such as optimizers, parameter initializations, and more have led to significant performance increases. This thesis aims to explore the concept of meta-learning to improve performance, through the often-overlooked component of the loss function. The loss function is a vital component of a learning system, as it represents the primary learning objective, where success is determined and quantified by the system's ability to optimize for that objective successfully.