A Survey of Deep Meta-Learning
This is a survey paper, so it is incremental, summarizing existing work for researchers in meta-learning.
The paper addresses the lack of a unified overview in the rapidly advancing field of deep meta-learning by providing a theoretical foundation and summarizing key methods categorized into metric-, model-, and optimization-based techniques, while identifying open challenges like performance evaluations on heterogeneous benchmarks and computational cost reduction.
Deep neural networks can achieve great successes when presented with large data sets and sufficient computational resources. However, their ability to learn new concepts quickly is limited. Meta-learning is one approach to address this issue, by enabling the network to learn how to learn. The field of Deep Meta-Learning advances at great speed, but lacks a unified, in-depth overview of current techniques. With this work, we aim to bridge this gap. After providing the reader with a theoretical foundation, we investigate and summarize key methods, which are categorized into i)~metric-, ii)~model-, and iii)~optimization-based techniques. In addition, we identify the main open challenges, such as performance evaluations on heterogeneous benchmarks, and reduction of the computational costs of meta-learning.