On the Robustness of Agentic Function Calling
This addresses robustness issues in autonomous LLM agents for real-world deployments, though it appears incremental as it builds on existing function calling benchmarks.
The paper tackles the problem of robustness in agentic function calling by introducing a benchmark that assesses resilience to naturalistic query variations and stability when toolkits expand with semantically related tools, identifying critical weaknesses in existing evaluation methodologies.
Large Language Models (LLMs) are increasingly acting as autonomous agents, with function calling (FC) capabilities enabling them to invoke specific tools for tasks. While prior research has primarily focused on improving FC accuracy, little attention has been given to the robustness of these agents to perturbations in their input. We introduce a benchmark assessing FC robustness in two key areas: resilience to naturalistic query variations, and stability in function calling when the toolkit expands with semantically related tools. Evaluating best-performing FC models on a carefully expanded subset of the Berkeley function calling leaderboard (BFCL), we identify critical weaknesses in existing evaluation methodologies, and highlight areas for improvement in real-world agentic deployments.