SPRIG: Improving Large Language Model Performance by System Prompt Optimization
This addresses the challenge of enhancing LLM performance efficiently for users by optimizing system prompts, though it is incremental as it builds on existing prompt optimization methods.
The paper tackled the problem of optimizing general instructions (system prompts) for large language models (LLMs) to improve performance across diverse tasks, proposing SPRIG, an edit-based genetic algorithm, and found that a single optimized system prompt performs on par with task-specific prompts, with further gains when combined, generalizing across models and languages.
Large Language Models (LLMs) have shown impressive capabilities in many scenarios, but their performance depends, in part, on the choice of prompt. Past research has focused on optimizing prompts specific to a task. However, much less attention has been given to optimizing the general instructions included in a prompt, known as a system prompt. To address this gap, we propose SPRIG, an edit-based genetic algorithm that iteratively constructs prompts from prespecified components to maximize the model's performance in general scenarios. We evaluate the performance of system prompts on a collection of 47 different types of tasks to ensure generalizability. Our study finds that a single optimized system prompt performs on par with task prompts optimized for each individual task. Moreover, combining system and task-level optimizations leads to further improvement, which showcases their complementary nature. Experiments also reveal that the optimized system prompts generalize effectively across model families, parameter sizes, and languages. This study provides insights into the role of system-level instructions in maximizing LLM potential.