TAPO: Task-Referenced Adaptation for Prompt Optimization
This work addresses the need for more domain-specific and adaptable prompt optimization in AI, offering a novel method for enhancing LLM performance on various tasks.
The paper tackles the problem of automated prompt optimization for large language models by introducing TAPO, a multitask-aware framework that incorporates task-specific characteristics, resulting in improved performance across six datasets.
Prompt engineering can significantly improve the performance of large language models (LLMs), with automated prompt optimization (APO) gaining significant attention due to the time-consuming and laborious nature of manual prompt design. However, much of the existing work in APO overlooks task-specific characteristics, resulting in prompts that lack domain specificity and are not well-suited for task-specific optimization. In this paper, we introduce TAPO, a multitask-aware prompt optimization framework composed of three key modules. First, a task-aware metric selection module is proposed to enhance task-specific prompt generation capabilities. Second, we present a multi-metrics evaluation module to jointly evaluate prompts from multiple perspectives. Third, an evolution-based optimization framework is introduced for automatic prompt refinement, which improves adaptability across various tasks. Extensive experiments on six datasets demonstrate the effectiveness of our approach, and our code is publicly available.