MODP: Multi Objective Directional Prompting
It addresses the challenge of optimizing prompt engineering for LLMs in practical applications, though it appears incremental by building on existing prompt engineering methods.
The paper tackles the problem of subjective and approximation-driven prompt engineering for large language models by introducing MODP, a framework that incorporates multi-objectivity and directional prompting, achieving a 26% performance gain on a summarization task and deploying it in a production tool used by over 10,000 support agents.
Recent advances in large language models (LLMs) have led to their popularity across multiple use-cases. However, prompt engineering, the process for optimally utilizing such models, remains approximation-driven and subjective. Most of the current research on prompt engineering focuses on task-specific optimization, while neglecting the behavior of the LLM under consideration during prompt development. This paper introduces MODP -- Multi Objective Directional Prompting, a framework based on two key concepts: 1) multi-objectivity: the importance of considering an LLM's intrinsic behavior as an additional objective in prompt development, and 2) directional prompting: a metrics-driven method for prompt engineering to ensure development of robust and high-precision prompts. We demonstrate the effectiveness of our proposed ideas on a summarization task, using a synthetically created dataset, achieving a 26% performance gain over initial prompts. Finally, we apply MODP to develop prompts for Dell's Next Best Action support tool, which is now in production and is used by more than 10,000 internal support agents and serving millions of customers worldwide.