Complexity-Guided Curriculum Learning for Text Graphs
This work addresses training optimization for text graph data, offering incremental improvements in curriculum learning methods.
The paper tackles training inefficiencies in text graph tasks by proposing a complexity-guided curriculum learning approach with a novel data scheduler, resulting in a model that gains more performance using less data and learns transferable curricula across GNN models and datasets.
Curriculum learning provides a systematic approach to training. It refines training progressively, tailors training to task requirements, and improves generalization through exposure to diverse examples. We present a curriculum learning approach that builds on existing knowledge about text and graph complexity formalisms for training with text graph data. The core part of our approach is a novel data scheduler, which employs "spaced repetition" and complexity formalisms to guide the training process. We demonstrate the effectiveness of the proposed approach on several text graph tasks and graph neural network architectures. The proposed model gains more and uses less data; consistently prefers text over graph complexity indices throughout training, while the best curricula derived from text and graph complexity indices are equally effective; and it learns transferable curricula across GNN models and datasets. In addition, we find that both node-level (local) and graph-level (global) graph complexity indices, as well as shallow and traditional text complexity indices play a crucial role in effective curriculum learning.