Reverse Prompt Engineering
This addresses a practical challenge in language model inversion for scenarios with strict resource constraints, offering a novel solution with potential applications in data generation.
The paper tackles the problem of reconstructing prompts from limited text outputs of a black-box language model, proposing a training-free framework that achieves high-quality prompt recovery and generates more semantically aligned prompts than state-of-the-art methods.
We explore a new language model inversion problem under strict black-box, zero-shot, and limited data conditions. We propose a novel training-free framework that reconstructs prompts using only a limited number of text outputs from a language model. Existing methods rely on the availability of a large number of outputs for both training and inference, an assumption that is unrealistic in the real world, and they can sometimes produce garbled text. In contrast, our approach, which relies on limited resources, consistently yields coherent and semantically meaningful prompts. Our framework leverages a large language model together with an optimization process inspired by the genetic algorithm to effectively recover prompts. Experimental results on several datasets derived from public sources indicate that our approach achieves high-quality prompt recovery and generates prompts more semantically and functionally aligned with the originals than current state-of-the-art methods. Additionally, use-case studies introduced demonstrate the method's strong potential for generating high-quality text data on perturbed prompts.