Dual-Criterion Curriculum Learning: Application to Temporal Data
This work addresses a bottleneck in Curriculum Learning for researchers and practitioners, offering a hybrid approach that is incremental but potentially applicable beyond time-series data.
The paper tackles the challenge of defining meaningful difficulty measures in Curriculum Learning by proposing a Dual-Criterion Curriculum Learning (DCCL) framework that combines loss-based and density-based criteria, showing improved performance over loss-only baselines and non-CL training in time-series forecasting tasks.
Curriculum Learning (CL) is a meta-learning paradigm that trains a model by feeding the data instances incrementally according to a schedule, which is based on difficulty progression. Defining meaningful difficulty assessment measures is crucial and most usually the main bottleneck for effective learning, while also in many cases the employed heuristics are only application-specific. In this work, we propose the Dual-Criterion Curriculum Learning (DCCL) framework that combines two views of assessing instance-wise difficulty: a loss-based criterion is complemented by a density-based criterion learned in the data representation space. Essentially, DCCL calibrates training-based evidence (loss) under the consideration that data sparseness amplifies the learning difficulty. As a testbed, we choose the time-series forecasting task. We evaluate our framework on multivariate time-series benchmarks under standard One-Pass and Baby-Steps training schedules. Empirical results show the interest of density-based and hybrid dual-criterion curricula over loss-only baselines and standard non-CL training in this setting.