Prompt and Parameter Co-Optimization for Large Language Models
This addresses the challenge of enhancing LLM performance for AI practitioners by synergizing prompt optimization and fine-tuning, though it is incremental as it builds on existing methods.
The paper tackles the problem of improving Large Language Models (LLMs) by jointly optimizing prompts and fine-tuning parameters, introducing MetaTuner, a framework that integrates both approaches to discover optimal combinations, with experiments showing it consistently outperforms baselines across diverse benchmarks.
Prompt optimization and fine-tuning are two major approaches to improve the performance of Large Language Models (LLMs). They enhance the capabilities of LLMs from complementary perspectives: the former through explicit natural language, and the latter through implicit parameter updates. However, prior work has typically studied them in isolation, leaving their synergistic potential largely underexplored. To bridge this gap, in this paper, we introduce MetaTuner, a novel framework that jointly integrates prompt optimization and fine-tuning for LLM training. Specifically, we introduce two neural networks to generate prompts and parameters, respectively, while allowing them to share a common bottom encoding layer to enable knowledge sharing. By the guidance of the final supervised signals, our framework is optimized to discover the optimal combinations between the prompts and parameters. Given that prompt learning involves discrete optimization while fine-tuning operates in a continuous parameter space, we design a supervised regularization loss to train our framework effectively. Extensive experiments across diverse benchmarks show that our method consistently outperforms the baselines.