Training Learned Optimizers with Randomly Initialized Learned Optimizers
This approach reduces research and engineering effort in machine learning by enabling self-training of optimizers, though it appears incremental as it builds on existing learned optimizer methods.
The authors tackled the problem of meta-training learned optimizers without relying on hand-designed optimizers by showing that a population of randomly initialized learned optimizers can train themselves from scratch in an online fashion, leading to rapid improvement through a positive feedback loop.
Learned optimizers are increasingly effective, with performance exceeding that of hand designed optimizers such as Adam~\citep{kingma2014adam} on specific tasks \citep{metz2019understanding}. Despite the potential gains available, in current work the meta-training (or `outer-training') of the learned optimizer is performed by a hand-designed optimizer, or by an optimizer trained by a hand-designed optimizer \citep{metz2020tasks}. We show that a population of randomly initialized learned optimizers can be used to train themselves from scratch in an online fashion, without resorting to a hand designed optimizer in any part of the process. A form of population based training is used to orchestrate this self-training. Although the randomly initialized optimizers initially make slow progress, as they improve they experience a positive feedback loop, and become rapidly more effective at training themselves. We believe feedback loops of this type, where an optimizer improves itself, will be important and powerful in the future of machine learning. These methods not only provide a path towards increased performance, but more importantly relieve research and engineering effort.