Differentiable Instruction Optimization for Cross-Task Generalization
This work addresses the challenge of instruction design for AI models to generalize across tasks, representing an incremental improvement over existing manual approaches.
The paper tackles the problem of identifying optimal instructions for cross-task generalization in instruction tuning by introducing learnable instructions optimized via bilevel optimization, resulting in improved generalization ability and enhanced instruction diversity compared to manual methods.
Instruction tuning has been attracting much attention to achieve generalization ability across a wide variety of tasks. Although various types of instructions have been manually created for instruction tuning, it is still unclear what kind of instruction is optimal to obtain cross-task generalization ability. This work presents instruction optimization, which optimizes training instructions with respect to generalization ability. Rather than manually tuning instructions, we introduce learnable instructions and optimize them with gradient descent by leveraging bilevel optimization. Experimental results show that the learned instruction enhances the diversity of instructions and improves the generalization ability compared to using only manually created instructions.