Auto-Prompt Generation is Not Robust: Prompt Optimization Driven by Pseudo Gradient
This work addresses robustness issues in auto-prompting for LLM users, but it is incremental as it builds on existing methods with a focus on noisy conditions.
The paper tackles the problem of robustness in automatic prompt generation by introducing PertBench, a benchmark for evaluating vulnerabilities, and proposes PGO, a gradient-free framework that uses pseudo-gradient signals to generate more robust prompts, showing consistent outperformance over previous methods under input perturbations.
While automatic prompt generation methods have recently received significant attention, their robustness remains poorly understood. In this paper, we introduce PertBench, a comprehensive benchmark dataset that includes a wide range of input perturbations, designed to systematically evaluate the robustness of current auto-prompting techniques. Our analysis reveals substantial vulnerabilities in existing prompt generation strategies, where even minor modifications to the prompt can lead to significant differences in model output. To address this issue, we propose PGO, a gradient-free prompt generation framework that leverages perturbation types as pseudo-gradient signals to guide LLMs in producing more robust prompts. In contrast to existing methods that assess prompt quality only on clean, well-structured inputs, our approach explicitly emphasizes robustness under noisy and perturbed conditions. Extensive experiments across diverse tasks and multiple LLMs show PGO consistently outperforms previous methods in maintaining performance under input perturbations.