InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models
This addresses the challenge of efficiently finding effective instructions for black-box LLMs, which is crucial for users relying on API-based models without access to internal gradients, though it is incremental as it builds on existing optimization techniques.
The paper tackles the problem of optimizing instructions for black-box large language models (LLMs) where backpropagation is not possible, by proposing InstructZero, which uses an open-source LLM to generate instructions from a soft prompt optimized via Bayesian optimization, and it outperforms state-of-the-art auto-instruction methods across various tasks.
Large language models~(LLMs) are instruction followers, but it can be challenging to find the best instruction for different situations, especially for black-box LLMs on which backpropagation is forbidden. Instead of directly optimizing the discrete instruction, we optimize a low-dimensional soft prompt applied to an open-source LLM to generate the instruction for the black-box LLM. On each iteration of the proposed method, which we call InstructZero, a soft prompt is converted into an instruction using the open-source LLM, which is then submitted to the black-box LLM for zero-shot evaluation, and the performance is sent to Bayesian optimization to produce new soft prompts improving the zero-shot performance. We evaluate InstructZero on different combinations of open-source LLMs and APIs including Vicuna and ChatGPT. Our results show that InstructZero outperforms SOTA auto-instruction methods across a variety of downstream tasks. Our code and data are publicly available at https://github.com/Lichang-Chen/InstructZero.