CITB: A Benchmark for Continual Instruction Tuning
This work addresses a practical problem for NLP researchers by providing a benchmark to study continual learning with instruction tuning, but it is incremental as it builds on existing paradigms without introducing new methods.
The paper tackles the problem of how instruction tuning works in continual learning tasks by establishing a Continual Instruction Tuning (CIT) benchmark with curated dialogue task streams, finding that existing continual learning methods do not effectively use instructions and sequential fine-tuning can yield similar or better results.
Continual learning (CL) is a paradigm that aims to replicate the human ability to learn and accumulate knowledge continually without forgetting previous knowledge and transferring it to new tasks. Recent instruction tuning (IT) involves fine-tuning models to make them more adaptable to solving NLP tasks in general. However, it is still uncertain how instruction tuning works in the context of CL tasks. This challenging yet practical problem is formulated as Continual Instruction Tuning (CIT). In this work, we establish a CIT benchmark consisting of learning and evaluation protocols. We curate two long dialogue task streams of different types, InstrDialog and InstrDialog++, to study various CL methods systematically. Our experiments show that existing CL methods do not effectively leverage the rich natural language instructions, and fine-tuning an instruction-tuned model sequentially can yield similar or better results. We further explore different aspects that might affect the learning of CIT. We hope this benchmark will facilitate more research in this direction.