SAGE: Stochastic Prompt Optimization via Agent-Guided Exploration
For practitioners of automatic prompt optimization, this work provides a framework and empirical comparison showing that agent-guided exploration can yield robust gains in open-ended dialogue tasks, though effectiveness is task-dependent.
The paper introduces SPO, a framework for stochastic prompt optimization, and compares three strategies including SAGE, a multi-agent pipeline. Across benchmarks, no single strategy dominates; SAGE achieves a statistically robust gain in next-day retention for a mental-health chatbot through eight cycles of A/B tests.
Context engineering has emerged as a primary lever for improving AI systems without parameter updates. Recent work showing that textual gradients do not function as real gradients motivates treating automatic prompt optimization (APO) as black-box search. We introduce SPO (Stochastic Prompt Optimization), a framework for stochastic search over prompt space, and compare three strategies of increasing sophistication: error-informed random search, a genetic algorithm with evolutionary operators, and SAGE (SPO via Agent-Guided Exploration), a multi-agent pipeline with diagnostic code execution. Across three benchmarks, no single strategy dominates; effectiveness depends on the interaction of landscape structure with error type. We further deploy SAGE on a mental-health chatbot under a continuous optimization paradigm, where it compounds eight cycles of individually-noisy A/B tests into a statistically robust gain in next-day retention. We argue that coupling qualitative diagnosis with quantitative validation is what makes agentic optimization effective for open-ended task-oriented dialogue.