SkillGraph: Graph Foundation Priors for LLM Agent Tool Sequence Recommendation
This addresses the challenge of tool sequence recommendation for LLM agents in structured workflows, offering a reusable prior to enhance performance, though it is incremental as it builds on existing retrieval and reranking methods.
The paper tackles the problem of LLM agents selecting and ordering tools from large API libraries, where existing semantic-only methods fail due to missing inter-tool dependencies, by introducing SkillGraph, a graph foundation prior mined from successful trajectories, which improves Kendall-τ from -0.433 to +0.613 on API-Bank and achieves Set-F1 = 0.271 on ToolBench.
LLM agents must select tools from large API libraries and order them correctly. Existing methods use semantic similarity for both retrieval and ordering, but ordering depends on inter-tool data dependencies that are absent from tool descriptions. As a result, semantic-only methods can produce negative Kendall-$τ$ in structured workflow domains. We introduce SkillGraph, a directed weighted execution-transition graph mined from 49,831 successful LLM agent trajectories, which encodes workflow-precedence regularities as a reusable graph foundation prior. Building on this graph foundation prior, we propose a two-stage decoupled framework: GS-Hybrid retrieval for candidate selection and a learned pairwise reranker for ordering. On ToolBench (9,965 test instances; ~16,000 tools), the method reaches Set-F1 = 0.271 and Kendall-$τ$ = 0.096; on API-Bank, Kendall-$τ$ improves from -0.433 to +0.613. Under identical Stage-1 inputs, the learned reranker also outperforms LLaMA-3.1-8B Stage-2 rerankers.