Linguistic and Argument Diversity in Synthetic Data for Function-Calling Agents
This addresses the problem of limited linguistic and argument diversity in synthetic data for function-calling agents, which is incremental as it builds on prior work focusing on other diversity aspects.
The paper tackles the challenge of obtaining high-quality diverse data for training function-calling agents by proposing a method that generates synthetic datasets optimizing diversity in queries and arguments, resulting in a 7.4% increase in accuracy on the BFCL benchmark compared to baselines.
The construction of function calling agents has emerged as a promising avenue for extending model capabilities. A major challenge for this task is obtaining high quality diverse data for training. Prior work emphasizes diversity in functions, invocation patterns, and interaction turns, yet linguistic diversity of requests and coverage of arguments (e.g., \texttt{city\_name}, \texttt{stock\_ticker}) remain underexplored. We propose a method that generates synthetic datasets via optimizing general-purpose diversity metrics across both queries and arguments, without relying on hand-crafted rules or taxonomies, making it robust to different usecases. We demonstrate the effectiveness of our technique via both intrinsic and extrinsic testing, comparing it to SoTA data generation methods. We show a superiority over baselines in terms of diversity, while keeping comparable correctness. Additionally, when used as a training set, the model resulting from our dataset exhibits superior performance compared to analogous models based on the baseline data generation methods in out-of-distribution performance. In particular, we achieve an $7.4\%$ increase in accuracy on the BFCL benchmark compared to similar counterparts.