Trajectory Supervision for Continual Tool-Use Learning in LLMs
For LLM practitioners seeking to improve tool-use learning, this work provides preliminary evidence that trajectory supervision may be beneficial, but the results are incremental due to limited experimental scope.
This paper investigates whether keeping tool-use trajectories during training improves continual learning of new API domains in LLMs. In a pilot study, retaining trajectory context boosted final exact full-call accuracy from 39.2% to 56.9% and API-name accuracy by 7.7 points, albeit with 25.1% more training tokens and a single seed.
Most language-model training data shows final artifacts, not the process that produced them. We study a tractable version of this question in tool use: when a model learns a stream of new API domains, does keeping tool-use trajectories help compared with stripping the intermediate API trace? We fine-tune Llama 3.1 8B Instruct with QLoRA on API-Bank using four sequential domain blocks. Condition A strips previous API request/response lines from the prompt and trains the model to predict the next API call. Condition B keeps the trajectory context. In a single-seed pilot, full held-out generation evaluation shows that Condition B reaches 56.9\% final exact full-call accuracy compared with 39.2\% for Condition A. B also improves final API-name accuracy by 7.7 points. However, B uses 25.1\% more training tokens, the run uses one seed, and the task is next-call prediction rather than full dialogue success.