Robust Prompt Optimization for Large Language Models Against Distribution Shifts
This work addresses a critical robustness issue for LLMs in practical applications, offering an incremental improvement over existing prompt optimization methods.
The paper tackles the vulnerability of automatic prompt optimization techniques to distribution shifts, such as subpopulation shifts in real-world scenarios like customer reviews analysis, by proposing a Generalized Prompt Optimization framework that incorporates unlabeled target data, resulting in significant performance improvement on the target group while maintaining comparable performance on the source group.
Large Language Model (LLM) has demonstrated significant ability in various Natural Language Processing tasks. However, their effectiveness is highly dependent on the phrasing of the task prompt, leading to research on automatic prompt optimization using labeled task data. We reveal that these prompt optimization techniques are vulnerable to distribution shifts such as subpopulation shifts, which are common for LLMs in real-world scenarios such as customer reviews analysis. In this light, we propose a new problem of robust prompt optimization for LLMs against distribution shifts, which requires the prompt optimized over the labeled source group can simultaneously generalize to an unlabeled target group. To solve this problem, we propose Generalized Prompt Optimization framework, which incorporates the unlabeled data from the target group into prompt optimization. Extensive experimental results demonstrate the effectiveness of the proposed framework with significant performance improvement on the target group and comparable performance on the source group.