An Empirical Comparison of Syllabuses for Curriculum Learning
This provides an empirical basis for choosing syllabuses in curriculum learning, though it is incremental as it compares existing methods without introducing major new paradigms.
The paper tackled the problem of comparing different syllabuses for curriculum learning, finding that syllabus choice has limited effect on generalization but task-dependent impact on learning speed, with the automated Predictive Gain approach performing competitively and the hand-crafted Look Back and Forward syllabus being best.
Syllabuses for curriculum learning have been developed on an ad-hoc, per task basis and little is known about the relative performance of different syllabuses. We identify a number of syllabuses used in the literature. We compare the identified syllabuses based on their effect on the speed of learning and generalization ability of a LSTM network on three sequential learning tasks. We find that the choice of syllabus has limited effect on the generalization ability of a trained network. In terms of speed of learning our results demonstrate that the best syllabus is task dependent but that a recently proposed automated curriculum learning approach - Predictive Gain, performs very competitively against all identified hand-crafted syllabuses. The best performing hand-crafted syllabus which we term Look Back and Forward combines a syllabus which steps through tasks in the order of their difficulty with a uniform distribution over all tasks. Our experimental results provide an empirical basis for the choice of syllabus on a new problem that could benefit from curriculum learning. Additionally, insights derived from our results shed light on how to successfully design new syllabuses.