Black-Box Tuning for Language-Model-as-a-Service
This addresses the challenge of fine-tuning language models in a black-box setting for users of LMaaS, offering a novel approach that is incremental but practical for real-world applications.
The paper tackles the problem of optimizing task prompts for large pre-trained language models accessed via black-box APIs where gradients are unavailable, proposing a black-box tuning framework using derivative-free optimization in a low-dimensional subspace, and shows it significantly outperforms manual prompts, GPT-3's in-context learning, and gradient-based methods like prompt tuning and full model tuning on RoBERTa with few labeled samples.
Extremely large pre-trained language models (PTMs) such as GPT-3 are usually released as a service. It allows users to design task-specific prompts to query the PTMs through some black-box APIs. In such a scenario, which we call Language-Model-as-a-Service (LMaaS), the gradients of PTMs are usually unavailable. Can we optimize the task prompts by only accessing the model inference APIs? This paper proposes the black-box tuning framework to optimize the continuous prompt prepended to the input text via derivative-free optimization. Instead of optimizing in the original high-dimensional prompt space, which is intractable for traditional derivative-free optimization, we perform optimization in a randomly generated subspace due to the low intrinsic dimensionality of large PTMs. The experimental results show that the black-box tuning with RoBERTa on a few labeled samples not only significantly outperforms manual prompt and GPT-3's in-context learning, but also surpasses the gradient-based counterparts, i.e., prompt tuning and full model tuning.