Less is More: Optimizing Function Calling for LLM Execution on Edge Devices
This addresses hardware and cost limitations for deploying LLM agents on edge devices, though it is incremental as it builds on existing function-calling methods.
The paper tackled the problem of inefficient function calling for LLMs on edge devices by selectively reducing available tools, resulting in up to 70% faster execution and 40% lower power consumption.
The advanced function-calling capabilities of foundation models open up new possibilities for deploying agents to perform complex API tasks. However, managing large amounts of data and interacting with numerous APIs makes function calling hardware-intensive and costly, especially on edge devices. Current Large Language Models (LLMs) struggle with function calling at the edge because they cannot handle complex inputs or manage multiple tools effectively. This results in low task-completion accuracy, increased delays, and higher power consumption. In this work, we introduce Less-is-More, a novel fine-tuning-free function-calling scheme for dynamic tool selection. Our approach is based on the key insight that selectively reducing the number of tools available to LLMs significantly improves their function-calling performance, execution time, and power efficiency on edge devices. Experimental results with state-of-the-art LLMs on edge hardware show agentic success rate improvements, with execution time reduced by up to 70% and power consumption by up to 40%.