Beyond Semantic Similarity: Reducing Unnecessary API Calls via Behavior-Aligned Retriever
This addresses the problem of cost and efficiency for users of tool-augmented LLMs, but it is incremental as it builds on existing retrieval-based methods.
The paper tackles the problem of inaccurate function calls in tool-augmented large language models, which cause inefficiencies and increased costs, by training a behavior-aligned retriever that provides behaviorally consistent demonstrations to help LLMs make more accurate tool-using decisions, resulting in significantly reduced erroneous function calls while maintaining high task performance.
Tool-augmented large language models (LLMs) leverage external functions to extend their capabilities, but inaccurate function calls can lead to inefficiencies and increased costs.Existing methods address this challenge by fine-tuning LLMs or using demonstration-based prompting, yet they often suffer from high training overhead and fail to account for inconsistent demonstration samples, which misguide the model's invocation behavior. In this paper, we trained a behavior-aligned retriever (BAR), which provides behaviorally consistent demonstrations to help LLMs make more accurate tool-using decisions. To train the BAR, we construct a corpus including different function-calling behaviors, i.e., calling or non-calling.We use the contrastive learning framework to train the BAR with customized positive/negative pairs and a dual-negative contrastive loss, ensuring robust retrieval of behaviorally consistent examples.Experiments demonstrate that our approach significantly reduces erroneous function calls while maintaining high task performance, offering a cost-effective and efficient solution for tool-augmented LLMs.