ProTIP: Progressive Tool Retrieval Improves Planning
This work addresses tool retrieval limitations in LLM-based planning, offering a novel method for dynamic toolboxes, but it is incremental as it builds on existing task decomposition approaches.
The paper tackles the problem of tool retrieval for multi-step planning with large language models by introducing ProTIP, a lightweight contrastive learning framework that implicitly performs task decomposition without subtask labels, achieving a 24% improvement in Recall@K=10 for tool retrieval and a 41% enhancement in tool accuracy for plan generation on the ToolBench dataset.
Large language models (LLMs) are increasingly employed for complex multi-step planning tasks, where the tool retrieval (TR) step is crucial for achieving successful outcomes. Two prevalent approaches for TR are single-step retrieval, which utilizes the complete query, and sequential retrieval using task decomposition (TD), where a full query is segmented into discrete atomic subtasks. While single-step retrieval lacks the flexibility to handle "inter-tool dependency," the TD approach necessitates maintaining "subtask-tool atomicity alignment," as the toolbox can evolve dynamically. To address these limitations, we introduce the Progressive Tool retrieval to Improve Planning (ProTIP) framework. ProTIP is a lightweight, contrastive learning-based framework that implicitly performs TD without the explicit requirement of subtask labels, while simultaneously maintaining subtask-tool atomicity. On the ToolBench dataset, ProTIP outperforms the ChatGPT task decomposition-based approach by a remarkable margin, achieving a 24% improvement in Recall@K=10 for TR and a 41% enhancement in tool accuracy for plan generation.