COSTAR-A: A prompting framework for enhancing Large Language Model performance on Point-of-View questions
This is an incremental improvement for developers using smaller, locally optimized LLMs on resource-constrained hardware.
The paper tackles the problem of inconsistent performance in smaller, fine-tuned LLMs when using the COSTAR prompting framework, by introducing COSTAR-A, which adds an 'Answer' component. The result shows that COSTAR-A enhances output structure and decisiveness for certain tasks, with the Llama 3.1-8B model exhibiting performance improvements compared to COSTAR alone.
Large Language Models (LLMs) are highly sensitive to prompt design, and making optimized prompting techniques is crucial for generating consistent, high-quality outputs. In this study, we introduce COSTAR-A, a novel prompt engineering framework that enhances the existing COSTAR method, which stands for Context, Objective, Style, Tone, Audience, and Response, by adding the 'Answer' component at the end. We demonstrate that while the original COSTAR framework improves prompt clarity and aligns outputs for larger LLMs, its performance is less consistent with smaller, locally optimized models, particularly in tasks that require more directive or constrained outputs. Through a series of controlled prompt-output assessments with smaller (at most 8 billion parameters), fine-tuned models, we found that COSTAR-A can enhance the output structure and decisiveness of localized LLMs for certain tasks, although its effectiveness varies across models and use cases. Notably, the Llama 3.1-8B model exhibited performance improvements when prompted with COSTAR-A compared to COSTAR alone. These findings emphasize the adaptability and scalability of COSTAR-A as a prompting framework, particularly in computationally efficient AI deployments on resource-constrained hardware.