Prompt Optimization Via Diffusion Language Models
This provides a model-agnostic and scalable approach for enhancing LLM performance through prompt refinement, addressing a bottleneck in prompt engineering for AI practitioners.
The paper tackles the problem of optimizing prompts for large language models by proposing a diffusion-based framework that iteratively refines system prompts using masked denoising, which consistently improves the performance of frozen target LLMs like GPT-4o-mini across diverse benchmarks such as τ-bench, SST-2, and SST-5.
We propose a diffusion-based framework for prompt optimization that leverages Diffusion Language Models (DLMs) to iteratively refine system prompts through masked denoising. By conditioning on interaction traces, including user queries, model responses, and optional feedback, our method enables flexible, span-level prompt updates without requiring gradient access or modifying the downstream language model. Across diverse benchmarks (e.g., $τ$-bench, SST-2, SST-5), DLM-optimized prompts consistently improve the performance of a frozen target LLM (e.g., GPT-4o-mini). We further show that moderate diffusion step counts provide the best balance between refinement quality and stability. These results highlight diffusion-based prompt optimization as a general, model-agnostic, and scalable approach for enhancing LLM performance through iterative prompt refinement.