Principled Curriculum Learning using Parameter Continuation Methods
This work addresses optimization challenges in deep learning, offering a theoretically justified approach that could improve training efficiency and model performance, though it appears incremental as it builds on existing curriculum learning concepts.
The authors tackled the problem of optimizing neural networks by proposing a parameter continuation method, which demonstrated better generalization performance than state-of-the-art techniques like ADAM in supervised and unsupervised learning tasks.
In this work, we propose a parameter continuation method for the optimization of neural networks. There is a close connection between parameter continuation, homotopies, and curriculum learning. The methods we propose here are theoretically justified and practically effective for several problems in deep neural networks. In particular, we demonstrate better generalization performance than state-of-the-art optimization techniques such as ADAM for supervised and unsupervised learning tasks.