Scholar Feed

Tool use / function calling

ToolLLM

ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

Heavily superseded#2 of 55 most-superseded · first seen Jul 31, 2023

Heavily superseded — a standard baseline that newer methods routinely beat

4 papers critique it · 3 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites ToolLLM as a baseline.

the fine-tuned models from datasets like ToolLLM~qin2023toolllm, ToolAlpaca~tang2023toolalpaca, and Gorilla~patil2023gorilla underperform in one (or more) of three key dimensions: (a) Generalizability: While the datasets are generated using diverse sets of APIs (e.g., ToolLLama uses RapidAPIs~{https://rapidapi.com/hub}, ToolAlpaca uses public APIs{https://github.com/public-apis/public-apis}, and Gorilla uses TensorFlow Hub, PyTorch Hub, and Hugging Face Hub), work from~basu2024apiblend has shown that models trained on these datasets have difficulty generalizing to out-of-domain datasets.
— Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks
“Unlike our work, these datasets are not conversational and just focus on mapping utterances to API calls, and they do not use intermediate structures (i.e., graphs) to ensure coverage and reduce hallucinations in generated tests.”
— Automated test generation to evaluate tool-augmented LLMs as conversational AI agents
“ToolLLM employs a tree-based scheme to minimize the number of tools required for task execution, yet it requires LLM calls across the entire tool set, making it impractical for edge devices -- where delay and power consumption are critical”
— Less is More: Optimizing Function Calling for LLM Execution on Edge Devices
“Unfortunately, in cases when the entire toolset needs to be enumerated, ToolLLM suffers from increased latency and energy consumption.”
— CarbonCall: Sustainability-Aware Function Calling for Large Language Models on Edge Devices

Beaten on benchmarks

Head-to-head results where a newer method reports beating ToolLLM. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.