A Flexible Multi-Agent LLM-Human Framework for Fast Human Validated Tool Building
This addresses the need for efficient, human-validated tool building in iterative problem-solving domains like scientific writing, though it appears incremental as a system-level integration of existing techniques.
The authors tackled the problem of creating tools for complex iterative tasks like scientific document generation by introducing CollabToolBuilder, a multi-agent LLM framework with human-in-the-loop guidance, which iteratively learns to build tools aligned with human intent while minimizing adaptation effort and feedback capture. Preliminary experiments showed it could generate state-of-the-art research papers or patents from abstracts.
We introduce CollabToolBuilder, a flexible multiagent LLM framework with expert-in-the-loop (HITL) guidance that iteratively learns to create tools for a target goal, aligning with human intent and process, while minimizing time for task/domain adaptation effort and human feedback capture. The architecture generates and validates tools via four specialized agents (Coach, Coder, Critic, Capitalizer) using a reinforced dynamic prompt and systematic human feedback integration to reinforce each agent's role toward goals and constraints. This work is best viewed as a system-level integration and methodology combining multi-agent in-context learning, HITL controls, and reusable tool capitalization for complex iterative problems such as scientific document generation. We illustrate it with preliminary experiments (e.g., generating state-of-the-art research papers or patents given an abstract) and discuss its applicability to other iterative problem-solving.