Attributing Structured-Output Gains in Function Calling: Interface Alignment versus Procedural Transfer
Provides a method for researchers to correctly attribute performance improvements in function-calling tasks, clarifying what structured-output scores actually measure.
The paper introduces a four-layer protocol to attribute gains in structured-output benchmarks to either interface alignment or procedural transfer. Applied to the Berkeley Function Calling Leaderboard, it finds that many gains are due to format compliance rather than transferable skill, with format-only prompts matching full skills in key cases and API-Bank gains matched within 0.5 pp by generic prompts.
Structured-output benchmarks reward both task decisions and interface compliance, so prompt-induced function-calling gains require attribution before they can be interpreted as transferable skill. We introduce a four-layer gain-attribution protocol for prompt-prepended skill injection, combining canonicalized rescoring, format-only controls, repaired/balanced induction, and portability checks. Applied to the Berkeley Function Calling Leaderboard (BFCL) and scoped with API-Bank, MATH-500, and MultiHop-RAG, the protocol shows that several apparent gains are better attributed to interface alignment than to procedural transfer: format-only prompts match or exceed full skills in key BFCL cells, repaired/balanced induction removes the largest sub-frontier gains, and API-Bank target-native gains are matched within 0.5 percentage points (pp) by length-matched generic procedural prompts. These findings treat format compliance as a useful engineering capability while clarifying what a structured-output score certifies. We release BFCL-CANONICAL and recommend canonicalized metrics, balanced induction, and format-only baselines for function-calling skill-gain attribution. Code and data are available at https://github.com/couragec/skill-injection-attribution.