EMO: Episodic Memory Optimization for Few-Shot Meta-Learning
This addresses the problem of optimization in few-shot learning for researchers and practitioners, but it is incremental as it builds on existing optimization-based meta-learning methods.
The paper tackles the challenge of few-shot meta-learning by proposing Episodic Memory Optimization (EMO), which uses gradient history from past tasks to guide parameter updates, resulting in improved performance and accelerated convergence on few-shot classification benchmarks.
Few-shot meta-learning presents a challenge for gradient descent optimization due to the limited number of training samples per task. To address this issue, we propose an episodic memory optimization for meta-learning, we call EMO, which is inspired by the human ability to recall past learning experiences from the brain's memory. EMO retains the gradient history of past experienced tasks in external memory, enabling few-shot learning in a memory-augmented way. By learning to retain and recall the learning process of past training tasks, EMO nudges parameter updates in the right direction, even when the gradients provided by a limited number of examples are uninformative. We prove theoretically that our algorithm converges for smooth, strongly convex objectives. EMO is generic, flexible, and model-agnostic, making it a simple plug-and-play optimizer that can be seamlessly embedded into existing optimization-based few-shot meta-learning approaches. Empirical results show that EMO scales well with most few-shot classification benchmarks and improves the performance of optimization-based meta-learning methods, resulting in accelerated convergence.