Adaptive Dependency-aware Prompt Optimization Framework for Multi-Step LLM Pipeline
This addresses the challenge of prompt optimization for users of multi-step LLM pipelines, offering a more effective and robust method, though it appears incremental as it builds on existing prompt optimization approaches.
The paper tackles the problem of optimizing prompts in multi-step LLM pipelines, where joint optimization is challenging due to dependencies and lack of supervision, and proposes ADOPT, a framework that models dependencies and uses gradient estimation to improve performance, achieving consistent outperformance over state-of-the-art baselines in experiments.
Multi-step LLM pipelines invoke large language models multiple times in a structured sequence and can effectively solve complex tasks, but their performance heavily depends on the prompts used at each step. Jointly optimizing these prompts is difficult due to missing step-level supervision and inter-step dependencies. Existing end-to-end prompt optimization methods struggle under these conditions and often yield suboptimal or unstable updates. We propose ADOPT, an Adaptive Dependency-aware Prompt Optimization framework for multi-step LLM pipelines. ADOPT explicitly models the dependency between each LLM step and the final task outcome, enabling precise text-gradient estimation analogous to computing analytical derivatives. It decouples textual gradient estimation from gradient updates, reducing multi-prompt optimization to flexible single-prompt optimization steps, and employs a Shapley-based mechanism to adaptively allocate optimization resources. Experiments on real-world datasets and diverse pipeline structures show that ADOPT is effective and robust, consistently outperforming state-of-the-art prompt optimization baselines.