Prompt-MII: Meta-Learning Instruction Induction for LLMs
This addresses the efficiency problem for users of large language models by reducing computational costs, though it is an incremental improvement over existing instruction induction methods.
The paper tackles the high inference cost of in-context learning for large language models by proposing PROMPT-MII, a reinforcement learning-based meta-learning method that generates compact instructions from training examples. The result is a 4-9 F1 point improvement (10-20% relative) in downstream model quality, matching in-context learning performance while using 3-13x fewer tokens.
A popular method to adapt large language models (LLMs) to new tasks is in-context learning (ICL), which is effective but incurs high inference costs as context length grows. In this paper we propose a method to perform instruction induction, where we take training examples and reduce them to a compact but descriptive prompt that can achieve performance comparable to ICL over the full training set. Specifically, we propose PROMPT-MII, a reinforcement learning (RL) based framework to meta-learn an instruction induction model that can generate compact instructions on the fly for an arbitrary new dataset. We train on over 3,000 diverse classification datasets from the HuggingFace hub, and evaluate on 90 unseen tasks. PROMPT-MII improves downstream model quality by 4-9 F1 points (10-20% relative), matching ICL performance while requiring 3-13x fewer tokens.