Graph-based Target Back-Propagation for Context Adaptation in Multi-LLM Agentic Systems
This work addresses the challenge of credit assignment and convergence in prompt optimization for multi-LLM workflows, offering a theoretically grounded and empirically superior solution.
GTBP introduces a graph-based back-propagation method for context adaptation in multi-LLM agentic systems, achieving consistent performance improvements over baselines across three benchmarks with comparable computational cost.
Context adaptation automates prompt engineering in LLM-based systems by iteratively revising tunable prompts from task feedback, without modifying model weights. Extending this paradigm to multi-LLM agentic systems is crucial: existing methods suffer from inaccurate credit assignment and lack convergence guarantees. We propose \textbf{G}raph-based \textbf{T}arget \textbf{B}ack-\textbf{P}ropagation (GTBP), a context adaptation framework for agentic workflows modeled as directed acyclic graphs. GTBP propagates local target outputs backward through the workflow graph and uses target--output discrepancies to guide a stage-wise prompt update mechanism. Theoretically, we show that GTBP's stage-wise prompt updates become stable over iterations, and that a sufficiently capable LLM optimizer can decrease the overall objective. Empirically, GTBP consistently outperforms strong baselines across three benchmarks while maintaining comparable computational cost.