Inductive-Deductive Strategy Reuse for Multi-Turn Instructional Dialogues
This addresses the challenge of aligning large language models with human expectations through high-quality instructional dialogues, representing an incremental improvement over existing methods.
The paper tackles the problem of generating diverse and in-depth instructions for multi-turn instructional dialogues by explicitly capturing complex rules from real dialogues, resulting in constructed dialogues that outperform competitive baselines on downstream chat models.
Aligning large language models (LLMs) with human expectations requires high-quality instructional dialogues, which usually require instructions that are diverse and in-depth. Existing methods leverage two LLMs to interact for automatic collection: one simulating a user to pose instructions, and the other acting as a system agent to respond. However, these user simulators struggle to model the rules behind how dialogues can pose different instructions without explicit guidance, resulting in general instructions. In this paper, we propose to explicitly capture the complex rules to help the user simulator pose diverse and in-depth instruction. Specifically, we first induce high-level instruction strategies from various real instruction dialogues serving as rules. Afterward, different possible strategies are applied to the newly given dialogue scenario deductively to pose various instructions. Experimental results show that our method can generate diverse and in-depth instructions. The constructed multi-turn instructional dialogues can outperform competitive baselines on the downstream chat model.