Automated Creation and Enrichment Framework for Improved Invocation of Enterprise APIs as Tools
This addresses the challenge of integrating enterprise APIs as tools for LLM agents, offering an incremental improvement in automation and accuracy.
The paper tackles the problem of poor documentation and complexity in enterprise APIs hindering LLM agents' tool selection and payload accuracy, proposing ACE, an automated framework that generates enriched tool specifications and dynamic shortlisting, improving accuracy by up to 25%.
Recent advancements in Large Language Models (LLMs) has lead to the development of agents capable of complex reasoning and interaction with external tools. In enterprise contexts, the effective use of such tools that are often enabled by application programming interfaces (APIs), is hindered by poor documentation, complex input or output schema, and large number of operations. These challenges make tool selection difficult and reduce the accuracy of payload formation by up to 25%. We propose ACE, an automated tool creation and enrichment framework that transforms enterprise APIs into LLM-compatible tools. ACE, (i) generates enriched tool specifications with parameter descriptions and examples to improve selection and invocation accuracy, and (ii) incorporates a dynamic shortlisting mechanism that filters relevant tools at runtime, reducing prompt complexity while maintaining scalability. We validate our framework on both proprietary and open-source APIs and demonstrate its integration with agentic frameworks. To the best of our knowledge, ACE is the first end-to-end framework that automates the creation, enrichment, and dynamic selection of enterprise API tools for LLM agents.