Supervisory Prompt Training
This addresses the scalability and efficiency issues in prompt engineering for LLMs, offering an alternative to fine-tuning, though it appears incremental as it builds on existing prompt optimization techniques.
The paper tackles the problem of manual and costly prompt engineering for Large Language Models by proposing Supervisory Prompt Training (SPT), which uses a dual LLM system to automate prompt generation and improvement, resulting in a 28.3% increase in accuracy for GPT-4 on GSM8K from 65.8% to 94.1%.
The performance of Large Language Models (LLMs) relies heavily on the quality of prompts, which are often manually engineered and task-specific, making them costly and non-scalable. We propose a novel approach, Supervisory Prompt Training (SPT). SPT automates the generation of highly effective prompts using a dual LLM system. In this system, one LLM, the generator, performs a task while the other, the corrector, provides feedback and generates improved prompts. In contrast to earlier techniques, both the generator and corrector collaboratively and continuously improve their prompts over time. We also introduce the concept of \textit{impact scores} to measure the sentence-level effectiveness of the prompts. Our method was tested on four benchmarks, testing the level of hallucinations in LLMs. Notably, we were able to increase the accuracy of GPT-4 on GSM8K from 65.8\% to 94.1\% (28.3\% increase). SPT advances LLMs by refining prompts to enhance performance and reduce hallucinations, offering an efficient and scalable alternative to traditional model fine-tuning.