Magnet: Multi-turn Tool-use Data Synthesis and Distillation via Graph Translation
This addresses the challenge of enhancing function calling in multi-turn conversations for AI agents, representing a strong specific gain in a domain-specific area.
The paper tackles the problem of limited performance of large language models in complex, multi-turn interactions involving multiple tools by proposing Magnet, a framework for synthesizing training trajectories, resulting in a 14B model that achieves 68.01 on BFCL-v3 and 73.30 on ToolQuery, surpassing the teacher model.
Large language models (LLMs) have exhibited the ability to effectively utilize external tools to address user queries. However, their performance may be limited in complex, multi-turn interactions involving users and multiple tools. To address this, we propose Magnet, a principled framework for synthesizing high-quality training trajectories to enhance the function calling capability of large language model agents in multi-turn conversations with humans. The framework is based on automatic and iterative translations from a function signature path to a sequence of queries and executable function calls. We model the complicated function interactions in multi-turn cases with graph and design novel node operations to build reliable signature paths. Motivated by context distillation, when guiding the generation of positive and negative trajectories using a teacher model, we provide reference function call sequences as positive hints in context and contrastive, incorrect function calls as negative hints. Experiments show that training with the positive trajectories with supervised fine-tuning and preference optimization against negative trajectories, our 14B model, Magnet-14B-mDPO, obtains 68.01 on BFCL-v3 and 73.30 on ToolQuery, surpassing the performance of the teacher model Gemini-1.5-pro-002 by a large margin in function calling.