PILLOW: Enhancing Efficient Instruction Fine-tuning via Prompt Matching
This work addresses the problem of high computational resource requirements for instruction fine-tuning, making it more accessible for individuals or small-scale entities, though it is incremental as it builds on existing LoRA and prompting techniques.
The paper tackles the challenge of achieving satisfactory performance with Low-Rank Adaptation (LoRA) for efficient instruction fine-tuning of Large Language Models by proposing PILLOW, a method that uses a discrimination-based prompting approach with a matching network and Reinforcement Learning, resulting in commensurate performance on various metrics while utilizing only consumer-grade GPU resources and significantly reducing computational costs.
Instruction fine-tuning has conventionally been employed to adapt Large Language Models (LLMs) to a variety of tasks. Nonetheless, this technique often necessitates substantial computational resources, making it impractical for deployment by individuals or small-scale entities. Recently, Low-Rank Adaptation (LoRA) has become a promising alternative, offering high capabilities on par with full tuning with reduced resource overhead. However, attaining satisfactory performance through the fine-tuning of LoRA is a non-trivial challenge. In this paper, we propose PILLOW, which aims to improve LoRA's performance by a discrimination-based prompting method, leveraging LLMs' In-Context Learning ability. PILLOW incorporates a matching network that selects prompts from a user-defined prompt pool, concatenates the selected prompts with the user instruction as input, and performs inference using the LoRA-fine-tuned LLMs. Trained with Reinforcement Learning, PILLOW exhibits commensurate performance on various evaluation metrics compared with typical instruction fine-tuning methods, utilizing only consumer-grade GPU resources and exhibiting a large reduction in computational costs.