Fine-tuning Large Language Models with Sequential Instructions
This addresses a limitation in instruction-tuned models for handling complex, multi-step tasks, though it is incremental as it builds on existing fine-tuning methods.
The paper tackles the problem of large language models struggling with queries containing multiple instructions by proposing sequential instruction tuning, which involves fine-tuning on data with chains of interrelated tasks, resulting in improved performance in coding, maths, and open-ended generation as evidenced by a new benchmark SeqEval.
Despite the success of existing instruction-tuned models, we find that they usually struggle to respond to queries with multiple instructions. This impairs their performance in complex problems whose solution consists of multiple intermediate tasks. Thus, we contend that part of the fine-tuning data mixture should be sequential--containing a chain of interrelated tasks. We first approach sequential instruction tuning from a task-driven perspective, manually creating interpretable intermediate tasks for multilingual and visual question answering: namely "translate then predict" and "caption then answer". Next, we automate this process by turning instructions in existing datasets (e.g., Alpaca and FlanCoT) into diverse and complex sequential instructions, making our method general-purpose. Models that underwent our sequential instruction tuning show improved results in coding, maths, and open-ended generation. Moreover, we put forward a new benchmark named SeqEval to evaluate a model's ability to follow all the instructions in a sequence, which further corroborates the benefits of our fine-tuning method. We hope that our endeavours will open new research avenues on instruction tuning for complex tasks.