From Generalist to Specialist: Improving Large Language Models for Medical Physics Using ARCoT
This addresses the challenge of adapting general-purpose LLMs for specialized domains like medical physics, offering a model-agnostic solution to reduce hallucinations and improve accuracy.
The study tackled the problem of applying large language models to specialized fields like medical physics by introducing ARCoT, a framework that improved domain-specific accuracy without fine-tuning, achieving up to 68% improvement and a 90% score on a medical physics exam.
Large Language Models (LLMs) have achieved remarkable progress, yet their application in specialized fields, such as medical physics, remains challenging due to the need for domain-specific knowledge. This study introduces ARCoT (Adaptable Retrieval-based Chain of Thought), a framework designed to enhance the domain-specific accuracy of LLMs without requiring fine-tuning or extensive retraining. ARCoT integrates a retrieval mechanism to access relevant domain-specific information and employs step-back and chain-of-thought prompting techniques to guide the LLM's reasoning process, ensuring more accurate and context-aware responses. Benchmarking on a medical physics multiple-choice exam, our model outperformed standard LLMs and reported average human performance, demonstrating improvements of up to 68% and achieving a high score of 90%. This method reduces hallucinations and increases domain-specific performance. The versatility and model-agnostic nature of ARCoT make it easily adaptable to various domains, showcasing its significant potential for enhancing the accuracy and reliability of LLMs in specialized fields.